Merge Meteor database in external MongoDB

Merge Meteor database in external MongoDB

So how could I merge the Meteor database in an external MongoDB? I’ve recently started to explore the possibilities of the Javascript app platform Meteor. I am using an external MongoDB and needed to merge the two together.

meteor-logo

First I installed Meteor and followed the tutorial on the website. A quite simple todo app which showed the capabilities of Meteor.

I wanted to build a (in my opinion) simple app which does the following:

  • Use a MongoDB already filled with pricing data
  • This data is harvested every day via a Node.js script
  • The harvest script does not run on the same server as Meteor
  • Show a textbox where the user can type a keyword and show the filtered data live as a table.

Although the above looked simple I ran in some problems:

Meteor uses its own database inside Mongo, often called “meteor”. In this database the defined collections are stored. However I had two databases: one with the app data and another with my pricing data. There were two options for merging these two:

  1. Merging the app data in the pricing database
  2. Merging the pricing database in the Meteor database

It turned out option 2 wasn’t working since the Meteor database is automatically bound to localhost. Localhost is not accessible from an external machineĀ  and therefore not by the harvest script eihter. So option 1 was the only one left.

The harvested prices database was on another machine, so I needed to move this one to the Meteor server. First I needed to export the data from the Meteor database. I used the backup feature of MongoDB called mongodump. Later I imported this data into my final database with mongorestore.

Import json files into MongoDB

Initially the prices were always stored in json files. To import these into my newly created MongoDB I used mongoimport which supports importing json files:

$ mongoimport --db pricingdata --collection prices --jsonArray --file prices-20150820.json

Now I had two databases: meteor and prices. In order to access the prices from my app I had to copy the items collection from prices into the meteor database.

Copy collectons between two MongoDB database

This article on Stackoverflow explains how to copy collections between MongoDB databases when they are on the same machine:

use pricing
db.items.find().forEach(function(d){ db.getSiblingDB('meteor')['items'].insert(d); })

Which actually iterates through the collection items and inserts them into the meteor database.

 

Leave a Reply

Your email address will not be published. Required fields are marked *