Much could be added to the Virtuemart forum dialogue referenced below, but it provides a snapshot of considerations and procedures that would assist in the development of an ecommerce website, particularly one of such great magnitude.
THE QUESTION OF MEGA ECOMMERCE
daviat: "Does anyone know the best and fastest way to add 300,000 products in virtuemart. I have a client whose site requires that but being a one man shop I am finding it impossible. Any ideas?" SOURCE
VIRTUEMART CSV IMPROVED
BanquetTables.pro: http://www.csvimproved.com/index.php?option=com_content&view=article&id=775:csvi-40-beta-2&catid=1:news
MY ANSWER
I haven't used CSV Improved on 2.0 yet but have found that performing the import in 2,000 to 4,000 product batches is safer and easier, especially if you have a lot of data loaded into each product, then I would certainly limit the amount of products per batch import. Test and backup before each import, ensuring you have an intuitive way to quickly identify the backups should anything go wrong.
For this project, I would recommend nothing less than a dedicated server. You want speed and security. Hopefully the client realizes the enormous undertaking you're to embark upon and aren't going cheap on the hosting through some shared service. I wouldn't be surprised to have multiple people assisting with the development of the product database. Either way, the biggest strain is going to be upon the MySQL database attempting to serve a product and category database of that magnitude, a database size I have not heard of in Virtuemart. The max I have reached is somewhere around 10,000 products, for which the eventually search engine traffic alone was an issue for the startup, not to mention the nominal marketing initiatives. Definitely limit the use and even disable some of the product/category listing modules, at least during development, since some tend to call all the products within a category and could well cause timeout errors.
Furthermore, you will want to ensure you have the products categorized in a legible manner to the user and the search engines. Syncing these products with the category structure prior to import can be tricky, which is why I will often create a single product by which I can manually plant in certain categories and then perform exports to easily and accurately copy and paste the product category path for several hundred or thousand products at a time. This makes it simple for me, since products may exist in several if not hundreds of categories, considering I create my own manufacturer categories and sub-categories within those. The VM manufacturer sort is too limited when it contains a several thousand products. The search engine has difficulty finding anything beyond page one, and with that many products, there is virtually 10-20 other pages of content the search engine cannot see. Good luck.
In closing, funding and project planning for such herculean effort should not be taken lightly.
Consider that at a development cost of a dollar per product, which is a very unlikely rate, the cost is $300,000. A more viable rate that is more apt to deliver a nominal return on investment (ROI) might be a cost of $10 to $20 per product, which results in a $3,000,000 to $6,000,000 ecommerce web development cost. We're talking big bucks. And with that, there ought to be ample research and planning performed prior to the initiation of development. From my perspective, such an effort is easily a two to five person effort. The lack of search engine optimized copy writing could quickly kill a project of such magnitude or at least translate the difference between earning $50,000 in gross sales per month versus $1,000,000. There are many other factors to also consider. I would want a lightening fast Microsoft Excel Spreadsheet expert (Open Office Calc) for starters.
My take, from a business perspective, such a venture ought to be taken up incrementally with earnings milestones met in order to initiate the next phase of development, so as to assess and adapt development strategy to capitolize upon market evolution. This would also enable for a residual ROI amidst development.
Another of the many considerations to take account of is the software. Can it technically sustain the demand that a project of this magnitude requires? And does such an open source ecommerce platform possess the life and direction that will deliver enough years of faithful service to earn the ROI to see your business venture into the next decade before having to migrate elsewhere?