diff / memory exhausted / udiff

diff / memory exhausted / udiff

  • Written by Walter Doekes

  • Published on: 2011-04-07

Sometimes when I’m unsure what a button in a database-driven application does, I simply click it and check the differences in the database before and after the click.

If the database dumped is (somewhat) sorted and with one line per row (for MySQL use –skip-extended-insert), this can be an easy method of verifying that your application action does exactly what you expect and nothing more.

Create a pre.sql before the click, click the button and watch the action happen. Finally, create a post.sql sql dump. Run diff(1) on the files, and voilà: a nice readable listing of state changes.

Sometimes however, the files can be too large for regular diff to handle. See this example:

$ ls -lh pre.sql post.sql
-rw-r--r-- 1 root root 286M 2011-02-23 16:55 post.sql
-rw-r--r-- 1 root root 286M 2011-02-23 16:50 pre.sql
$ time diff pre.sql post.sql
diff: memory exhausted

real  0m15.236s
user  0m0.132s
sys 0m2.480s

Ouch! Not enough memory. This is where udiff (microdiff or unified-diff, whichever you prefer) comes in:

$ time ./udiff pre.sql post.sql >tmp.diff

real  0m16.878s
user  0m15.205s
sys 0m1.372s

And you’re left with a nice readable diff:

$ cat tmp.diff
--- pre.sql 2011-02-23 16:50:38.000000000 +0000
+++ post.sql  2011-02-23 16:55:03.000000000 +0000
@@ -444245,7 +444245,6 @@
 INSERT INTO `billing...
 INSERT INTO `billing...
 INSERT INTO `billing...
-INSERT INTO `billing... (censored values)
 INSERT INTO `billing...
 INSERT INTO `billing...
 INSERT INTO `billing...
@@ -872067,6 +872066,8 @@
 INSERT INTO `django_admin_log` VALUES (4378,'2011-02-23 12:35:20'...
 INSERT INTO `django_admin_log` VALUES (4379,'2011-02-23 12:35:31'...
 INSERT INTO `django_admin_log` VALUES (4380,'2011-02-23 12:37:08'...
+INSERT INTO `django_admin_log` VALUES (4381,'2011-02-23 16:53:26'...
+INSERT INTO `django_admin_log` VALUES (4382,'2011-02-23 16:53:31'...

@@ -872226,7 +872227,7 @@

 LOCK TABLES `django_session` WRITE;
 INSERT INTO `django_session` VALUES ('d6b...
-INSERT INTO `django_session` VALUES ('8d7...
+INSERT INTO `django_session` VALUES ('8d7...
 INSERT INTO `django_session` VALUES ('091...

Convinced? Get the udiff python application (view) and the test suite Makefile (view) now.

Back to overview Newer post: linux / canon mf8350 / printer driver Older post: pcap / capture fragments / udp