Iterate from a Doctrine_Query

Question

I am using Doctrine 1.2 in a symfony project.

I would like to iterate with doctrine, I have a Doctrine_Query object, I want to iterate on the "result set" of this query because I can have 100K records and don't want to run out of memory.

Something like :

$it = $query->iterate();
while($it->next()) {
  $obj = $it->getOjbect();
  //Do some stuff with this object
}

I figured out how todo by using a custom hydrator.

For a symfony project, in ProjectConfiguration.class.php :

public function configureDoctrine(Doctrine_Manager $manager)
{
  $manager->registerHydrator('my_hydrator', 'Doctrine_Hydrator_MyHydrator');
}

In a controller :

$this->filename = $query->execute(array(), 'my_hydrator');

Added in lib folder : 

class Doctrine_Hydrator_MyHydrator extends Doctrine_Hydrator_Abstract
{
    public function hydrateResultSet($stmt)
    {
      $filename = sfConfig::get('sf_data_dir').'/'.time().'.csv';
      $fh = fopen($filename,'w');
      while($f = $stmt->fetch(Doctrine_Core::FETCH_BOTH)){
        $f = array(
            $f[0],
            $f[1],
            $f[2],
            $f[3],
            $f[4],
            $f[5],
            $f[6],
            $f[7],
            $f[8],
        );
        fputcsv($fh, $f); 
      }
      fclose($fh); 
      return $filename;
    }

adlawson adlawson · Accepted Answer · 2011-07-15T21:37:53

If you're concerned about running out of memory, you should look into returning the results as arrays instead of objects.

This will of course mean you can't then use the results as objects (so you can't do $row->save()), which for 100k records, you probably shouldn't need anyway

From the documentation:

$q = Doctrine_Query::create()
->select('u.id, u.username, p.phonenumber')
->from('User u')
->leftJoin('u.Phonenumbers p');

$results = $q->execute(array(), Doctrine_Core::HYDRATE_ARRAY);

Iterate from a Doctrine_Query

1 Answers