Archive for August, 2010

Magento Delayed Migration

Wednesday, August 25th, 2010

Weird title, I know. Here is the synopsis, you are running a Magento store and build a “sandbox” copy of the server so people can add/edit items and you can “push” them to the live server every 45 days.

Well this was far tricker than I originally thought, first I copied the db and the front-end. No problems there, now everything is the same and they can begin editing. Next they added 1k products to the sandbox, but also updated a bunch of information in the live server. So I exported all items from the live server and imported them in to the sandbox, now the sandbox has all the current updates and the new items.

1) Exported this and imported in to the live server. Crashed mysql ruined this and all my other magento stores.
2) Exported this and split in to 10 files of 200 records each and began importing, broke magento and all my other magento stores.
3) Exported, split, edited and created my own command line import program, success, partially. It imported the base information no options, no images, no cross-sell data, etc….
4) Exported a fresh copy of the sandbox with the updates, then exported a fresh copy of the live server, imported both in to new dbs. From there I deleted all catalog_ tables, all eav_ talbes, all rating tables, all review tables, all salesrule tables, all tag tables and the core_url_rewrite table. Once this was done, I copied the images from one server to the other with the -u option so only new or updated images would copy to the /media/catalog/products etc… finally I exported the tables from the sandbox db that I had just truncated in the live db, and viola … worked like a charm

Magento hates everybody, and I hate magento!!

Thursday, August 12th, 2010

So we’ve put up store after store using the magento open source e-commerce platform. I’ve had magento 1.2.1.1 – 1.4.1.1 and none of them are worth a damn when you try to import products.

At this point I’m writing a new import process that will still handle all the EAV crap, and indexing etc… maybe even use the Mage structure, but it’s going to be faster.

For now here are a couple of things you can do to increase the import process.

1) Truncate all log files, doesn’t seem like it should be tied together but whatever.
2) Truncate table datatflow_batch_export and dataflow_batch_import
3) Truncate table report_event (this is the event log basically, so you will mostl likely have these in your var/reports folder also.
4) remove cache and session values from the system, locate in var/session/ and var/cache/

I’ll update when I’ve written some new code, it’s going to be for 1.3.2.4 and 1.4.1.1 so keep an eye out.

Couple of pics I found that are fun

Thursday, August 12th, 2010


Take a look