Problems We Had Using MeteorJS for Web DevelopmentCreated: April 13, 2017
This is just my experience and MeteorJS might be a perfect fit for some applications. I am also might be noob when it comes to all this so maybe I only experienced these issues because of my lack of knowledge. If so feel free to post a comment and teach me something. It’s been about 2 months + since I’ve used Meteor and since then, there have been new releases may be addressing some issues which I am unaware of, so bear that in mind. This is also a slightly shallow discussion cause it’s very hard to into complete detail about these issues and other surrounding issues without writing a damn 20+ page mini book. Anyways I don’t think I’m doing this topic proper justice so any criticisms are welcome.
Over the last year, CloudWaitress was being developed as a prototype using MeteorJS and I have learnt a significant amount regarding the strengths and weakness of this framework. Since then we have changed our stack significantly due to some of the technical difficulties that arose as we neared production while using Meteor.
Below is a description of some of the scenarios encountered during the development of CloudWaitress where Meteor was a significant bottleneck. I also talk about how these problems were solved. Funny enough when I first started using Meteor, I had not considered a large amount of these things and how it would have an impact on the application. So, without further to do…
Terrible Database Performance Compared to Node Mongo Adaptor
The official database for Meteor is Mongo. While I have not comprehensively tested read and write performance of MeteorJS using Mongo compared to a Node Mongo Driver, I did conduct some scenario specific testing due to some technical issues.
With CloudWaitress, customers can create option sets which represent dish customization possibilities. An option set “Sauce” for example can contain options such “Tomato”, “Chill” and even “Super Special” for an extra $2. These options sets can then be applied to dishes where they can be further customised per dish.
Thus, this requires a data structure where an option set object is created, stored and then re-created under each dish inside an array of available option sets for that dish. A user can then customise the global option set object and modify an individual dishes option sets if required.
Now when someone wants to update an option set, the program must look for all the dishes that contain an identical copy of the option set and update that along with the global option set object itself.
This means writing a mongo update that scans for the dish and option set _id’s under each dish to find the set that must be modified and then update it to the new object.
When I ran this in Meteor with about 80 dishes created each with several option sets, my computer would hang and take about 15+ seconds to process this. A galaxy hosted Meteor instance needed at least 2.0 ECU to handle this function without crashing and restarting which costs a lot to run. Even then it would essentially hang the application. Why this happens I honestly don’t know. I tried so many things to fix it but nothing worked. Granted this test was run with a schema validator in place but mostly everything else was stripped away so I can’t see this making a huge difference given the schema is extremely light.
The exact same thing done using mongojs, a simple utility npm library over the node mongo driver package performed a 100x better. On a free Heroku dyno, a HTTP POST request from the client to the REST server running mongojs completed virtually instantly with no stress.
It’s possible that I just suck and this whole thing should be structured differently and then this wouldn’t even be an issue. Or maybe there are other factors that I am unaware of that are playing such a significant role in this. That said, it shouldn’t be the case that Meteor should crash a powerful server or even hang my computer to make this function happen.
After this, it seemed that the only safe way forward was just to run a dedicated REST server and offload all data storage, retrieval and what not to this instead of doing it within the Meteor application. Doing this does break some part of how Meteor works, now that the data isn’t going through Meteor, the UI no longer updates optimistically and instead needs to sync with the database which can take a few seconds. The simple work around to this problem is to create an API function that updates your local collection on top of sending the request to the REST server for processing. If something happens, the data can be reverted and an error message displayed. Keep in mind its best to avoid this for critical information that users should be 100% aware of what the state of the data is. You don’t want someone thinking they have saved something very important only to discover something went wrong and the request wasn’t processed correctly. It's best to wait in these situations for the REST server to respond with a success which is often incredibly fast anyways.
Difficult to Get Server Side Rendering, Fast Page Loads & SEO Done Correctly
With CloudWaitress, our users can create their own custom online food ordering store. To enable maximum SEO potential, it was necessary that the all the HTML content be rendered to the client in the initial request with the appropriate SEO tags. This was also ideal for maximum page load speed.
A user’s store sits on a subdomain of cloudwaitress.com or at their own domain of choice which is then pointed to our servers. The way the setup worked was when a client requested one of our store URL’s, this would resolve to our Meteor server which would respond to the client with the Meteor application.
With Meteor, the problem is that before a Meteor application renders on the client, it must load up the Meteor client side JS bundle along with the initial HTML file. This bundle (around 700kb for the store's application) is then executed on the client. From here the Meteor client then had to find the appropriate store from the database based on the URL, fetch the data from the server, create the custom CSS stylesheet for the theming and render everything appropriately. This all happens on the client. All in all, my Oneplus One mobile (medium-end device) took on average about 12-15 seconds to load it. My desktop could do it in about 6-8 seconds. Not a very ideal situation to have.
The next problem is because the page takes so long to load google absolutely hates it especially since the initial HTML contains no content. Granted these aren’t issues with just Meteor as much as it is an architectural issue that any SPA framework may encounter.
While I cannot remember exactly how some of the proposed solutions for these issues worked, there were a few options. First there was fast render, which tries to emulate SSR but in my opinion, it really worked with mixed results. Even though I did the validation test to ensure it was working, it made little difference. Maybe I just sucked at implementing it. The actual SSR routing package was iffy with very limited support and capability so that was a no go.
Even using pre-render didn’t really help with SEO. I tested this initially using a plain Meteor app to host the static front CloudWaitress website. I created the site, put it up, created a google webmaster account for it, submitted it to the google index. 3 weeks later, if you googled “cloudwaitress”, the website only showed up on the 2nd or 3rd page with the meta description and tag incorrectly. I validated that pre-render was integrated correctly but even then, google was not happy. When I remade it as a fully static site and hosted it normally, within 1 week, it was on the 1st page of google when you searched “cloudwaitress” with the tags showing correctly. That was enough proof for me that this was not going to work. I did see other people using Meteor who could rank fine but I have no idea why this didn’t happen for me properly.
The new setup we moved to for production is now significantly better. Currently, we are using ExpressJS and React to make it happen. ReactJS allows you to pre-compile your SPA templates in the server with your data and then render it in the initial request. Now everything can be done in one request to the server which will then validate the domain, fetch the data, compile the custom CSS, SEO tags and everything into the initial HTML and send it down in one shot. The result is a page that loads in about 3-4 seconds flat on any device. React still sends down a client side bundle which must mount itself before your event handlers work. This compared to Meteor's bundle is much smaller and significantly faster. The added advantage of this setup is there is much more control over the whole request and response. This means you can easily do any action in a much more time, for example compiling the CSS in the server as opposed to the client.
Meteor’s Biggest Strength is Its Weakness
For us at CloudWaitress, we found Meteor did a fantastic job in many areas but where it failed us, mainly due to being the wrong tool for the job or too opinionated, it always became a massive burden. When you use Meteor, you are essentially locked in. While there is NPM compatibility which is a huge plus, if you have an issue with the core of how Meteor works, the only way out is to move away from Meteor which in turn requires a nearly complete application port to a new stack. With the experiences we had, there was no way around but to pick a different tool for the job.
The issue with Meteor is it tries to do everything, it tries to provide you with all the core components required for a modern web application. Unfortunately, because it’s all bundled up together in some semi-interdependent way, it’s not easy to just swap out what doesn’t work. Furthermore, the project is so large and complex that an average joe dev like me can’t just dive into and fix what I need. So the only hope then is that your issue is on the Meteor development roadmap and is going to be addressed soon.
Your software in turn ends up being at the mercy of others which is not what you want as a tech company. This leads to the next problem, all the rapid updates which fix issues requiring you to go through the scary process of upgrading your application version hoping nothing major breaks. For an application in production, this is a very real threat. There have been multiple occasions where updating would mean the app wouldn’t compile properly or something needed to be fixed.
For a production application, you want stability and predictability. In my experience, this seems to be much more achievable when each component can function as an individual unit that can be used anywhere and easily maintained by other developers. If you look at how an alternative Meteor DIY build could look like, it’s some server package (say ExpressJS), with react + redux, socket.io for real-time coms, some database adaptor, etc. Most of these are separate packages that are maintained individually and function individually. They are also smaller, easier to understand and encompass a smaller scope. That means fewer bugs and issues, with greater stability and predictability. They can also easily be stacked and connected with each other. Best of all, if something doesn’t do the job correctly, you simply swap out that one part, not your entire stack. The benefits this approach has are so significant.
So, What’s the Takeaway?
Think about exactly what your application needs to achieve, what the specific tools for doing those jobs are, and how you can join them together. Meteor is great but once you use it, you are at the mercy of it and swapping it out requires nearly changing your whole stack and how you did things before.