This is a naive PHP script I wrote to migrate data from mysql to mongo DB. I have tested transferring 10 million rows with no problem. Mongo and mysql DB credentials are not handled in this code. Even though code doesnot handle many mongo exceptions, it is good enough for normal use cases. Anybody is welcome to refine the code.
<? // Usage ./mysqltomongo.php mysqldb mysqltable mongodb mongocollection function mongo_connect($db,$collection) { $m = new Mongo(); //handle username,ports and passwords $mydb = $m->$db; if(!isset($collection)) return $mydb; else { $mycollection = $mydb->$collection; return $mycollection; } } $mysqldb=$argv[1]; $mysqltablename=$argv[2]; $mongodb=$argv[3]; $mongocollection=$argv[4]; $link = mysql_connect(...); //give proper info if (!$link) { die('Could not connect: ' . mysql_error()); } mysql_select_db($mysqldb); $result = mysql_query("SELECT * from ".$mysqltablename); $fields = mysql_num_fields($result); $coltypes=array(); for ($i=0; $i < $fields; $i++) { $coltypes[mysql_field_name($result, $i)]=mysql_field_type($result, $i); } $mdb = mongo_connect($mongodb); $mdb->dropCollection($mongocollection); $collection= mongo_connect($mongodb,$mongocollection); while ($row = mysql_fetch_array($result, MYSQL_ASSOC)) { $newrow=transform($row,$coltypes); try { $collection->insert($newrow); } catch(MongoException $e) { print_r($e); } } function transform($row,$coltypes) { $val; $ret=array(); foreach($row as $k=>$v) { $val=utf8_encode($v); if($coltypes[$k]=="real") $val=floatval($val); else if($coltypes[$k]=="int") $val=intval($val); $ret[$k]=$val; } return $ret; } mysql_close(); ?>
Row 25 should be
ReplyDelete$result = mysql_query("SELECT * from $mysqltablename");
instead of
$result = mysql_query("SELECT * from ".$mysqltablename);
I am using the same script you have posted over here . But i am getting the following error "
ReplyDeleteFatal error:Out of memory (allocated 1112276992) (tried to allocate 7200000 bytes) in (directory name goes here