Drupal node_save speed up

Question

I have to insert millions of nodes programmatically (import data from xml). Node has many cck fields such as photos, votes, node references and others. Is there some tips to speed up this process (nodes are saved through drupal api node_save)?

To be clear what I'm trying to find, here is an example: we don't need to clear cache during data import, so we can comment last line in node_save function.

Any other useful performance tips?

Answer 1

I don't know for which purpose you need to create millions of nodes programmatically, but there is module exist Devel and its documentation on Document.

I'm not sure about how many node can be created at once by using this module. But it worthy to test in your case.

Check Generate content which can :

"Accelerate development of your site or module by quickly generating nodes, comments, terms, users, and more."

Answer 2

I had a solution for you

First : you have to parse your .xml file into .csv "comma delimiter" . you can use a lot of tools to do this , you don't have to write the code . have a look at this sites

1- http://codestips.com/php-xml-to-csv/ .

2- http://www.w3schools.com/php/func_filesystem_fgetcsv.asp.

3- Convert large XML file to CSV in PHP .

Second : Install drupal module note import " http://drupal.org/project/node_import "

in node import you'll import csv file and in configuration you'll set which type you need to create by this file . Also you have to define csv fields reference to the chosen content type . this module will generate node for each csv record. ex if it contains 100 row then 100 node will be created . this link is the step by step guide " http://drupal.org/node/827750 "

another solution :

1- Is to create content type called Import and add file field upload (to upload your xml file) then create only one instance .

2- Then hook on node_api "operation submit" add your custom code to parse the uploaded xml and iterate on number of elements you have in the file

3- Finally, create node programmatic and set it's value with the xml element child value .

Code for create programmatic note :

$node = new stdClass();
//Main Node Fields
$node->type = 'video'; //This can be any node type
$node->created = time();
$node->changed = $node->created;
$node->promote = 0; // Display on front page ? 1 : 0
$node->sticky = 0;  // Display top of page ? 1 : 0
$node->format = 2;  // 1:Filtered HTML, 2: Full HTML
$node->status = 1;  // Published ? 1 : 0
$node->language = 'en';

/* Custom Fields of Video */
$node->title = $node->name = "string";
$node->field_video_id[0]["value"] = "string";
$node->field_updated[0]['value'] = "string";
// this code must be placed to save the created node
if ($node = node_submit($node)) {
node_save($node);

drupal_set_message(t("Node " . $node->title . " added correctly"));
} else {
drupal_set_message(t("Node " . $node->title . " added incorrectly"), "error");
}

Contact me for further information .