Using Hash Value (track_changes) To Detect Source Data Changes in Migrate for Drupal 7

When you import data with Migrate (http://drupal.org/project/migrate) it is nice to have ability to update already imported data if source has changed. To track updated data Migrate uses highwater marks (more on this: http://drupal.org/node/1223936). Highwater mark is a column of a source data which has timestamp of the last data change. However, the problem with highwater mark is that you don't always have such "time tracking" enabled on source's page. For example, if you migrate users from D6 to D7, there is no field in {users} table, which stores "changed" timestamp, like in {node} table for example.

So what to do? Luckily, Migrate 2.6 beta 1 came out ten days ago which brings us hash value based tracking: http://drupal.org/node/1997616. Now you don't need timestamps/highwater marks. Now, Migrate will automagically create hash value for every row of your source data and will compare it against already stored hashes, if hashes aren't equal, it will treat row data as updated an update it. Pretty neat!

To enable this option, all you need to do is to add 'track_changes' => 1 to $options before you map source in your migrate class constructor, it may look like this:

class ExampleNodeMigration extends Migration {
  public function __construct($arguments) {
    parent::__construct($arguments);
   $options = array('track_changes' => 1);
   $this->source = new MigrateSourceSQL($this->query(), $this->fields(), NULL, $options);
}

Update: put this snippet on dropbucket.org: http://dropbucket.org/node/626

After this, during the first import run Migrate will auto-update all content you have (as if you've forced update option) and create hash values for every row. During next runs, it will import new content and update content that has changed at source.

Hope this tutorial will be helpful, since there is no documentation yet for this really usefull feature.

Comments

Submitted by tenken on Tue, 2013-05-28 21:21

this is actually really cool.

Submitted by kalabro on Wed, 2013-05-29 01:11

Very cool new feature! Thanks for sharing!

Add new comment

You are here