Moving to Webpack 5

It’s been 2 weeks since the coding period started, and it’s time for a rundown of things that I have done and hope to do in the next couple of weeks.

Note: This is my 2nd Bi-Weekly blog. You can find the rest of them here

Summary

For those in a hurry - here is a list of the Pull Requests I submitted in the past couple of weeks.

Issue(s)	Description	Pull Request	Status
#4510	Replace react-motion with react-spring	#4992	Merged
#4633	Upgrade to webpack version 5	#4982	Merged
-	Fix JS warnings for server rendered routes	#4997	Merged
-	Switch to Istanbul for coverage	#5002	Merged
-	Update coverage documentation	#5018	Merged
#5003, #5005	add support for HMR	#5013	Merged

Now, let’s go into a bit more detail.

Webpack 5

Upgrading to Webpack 5 was always going to be a bit painful. While the webpack configuration itself wouldn’t need to change much(although I did end up refactoring a major part of it), the plugins, loaders, and packages we used alongside webpack needed to be compatible with Webpack 5 for things to go smoothly. Unfortunately, that wasn’t the case for a lot of them. Parallel Webpack was probably the most problematic package and we had to get rid of it altogether.

After fixing these compatibility issues and getting the build to pass, I did some benchmarking.

The initial performance was much worse than I expected. After some debugging, I found that the plugin taking the most time during the build was the ESLint webpack plugin. I’ve made multiple attempts to fix this but had no luck. In the end, I ended up setting lintDirtyModulesOnly to true in the plugin configuration. This meant that the plugin would only cache files that were “dirty” - that is, files that had ESLint errors.

This isn’t a permanent fix but it did improve the build times considerably. Here are some numbers -

command	Cache Exists?	Webpack 5	Webpack 4
yarn start	no	20s	40s
yarn start	yes	10s	10s
yarn build	no	58s	75s
yarn build	yes	41s	25s

React Spring

We used React Motion to interpolate values between 2 points in a spring-like manner. This is different from time-based animations, where the values change linearly(or according to a Bézier curve)

React Motion however had been unmaintained for quite a while now. So I swapped it out for React Spring. React Spring is a lot more powerful than React Motion - but we only used it for interpolating values - the actual drag and drop was implemented using React DnD

I also added a new drop target to make it possible to drag items into an empty week.

Here’s how it looked in the end!

Istanbul for Coverage

The way we generated coverage reports earlier was far from ideal.

1. The reports were based on the webpack bundle and so were quite far away from the actual source code

2. We used a regular expression to break up the main bundle into multiple files to get somewhat readable coverage reports. This was a hack since we relied on comments and code inserted by Webpack. A change in that would(and indeed did) break the entire thing.

3. The instrumentation happened in the middle of the test suite and had to rely on a different bundle than the one used during production.

The main reason we did things this way was because JSCover, the instrumentation library we used, didn’t support source maps. Fortunately, there was a good alternative - Istanbul. It did support source maps so almost all of the problems I mentioned earlier didn’t exist.

Unfortunately, Istanbul also brought along its problems. For one, it didn’t store the coverage report in local storage. This meant that we had to manually extract the coverage data after each test. It also generated considerably larger reports - around 7MB - so storing that in the local storage manually wasn’t an option either. Initially, I was writing a new file containing the coverage report after each test and merging them at the end. But given that each file was 7MB, and we had hundreds of tests, doing that was simply impossible.

The way I ended up solving this was by keeping at most 2 coverage files in the folder at the same time. After each test, it would merge the coverage report of the current test with one of the previous tests.

Istanbul also has a million different ways to instrument your code. You have multiple webpack plugins, a command line utility, a babel plugin and probably some more novel ways of essentially doing the same thing.

Unfortunately, only the babel plugin worked for me. Others didn’t because of the differences between ES5 and ES2017 - merging coverage reports with different EcmaScript versions led to data loss. Because the babel plugin was run before the transpilation step, it didn’t have this problem.

I don’t take credit for figuring this out - see why is there sometimes a discrepancy between functions and function Map? for more details.

Hot Module Replacement

Hot Module Replacement allows code to be updated at runtime without the need for a full refresh.

Say you’re on a page, and you have changed some state - say incremented a counter. Without HMR, when you change some code, the page would refresh and the counter would reset to 0. With HMR, the page would no longer refresh and the counter would maintain its state. This makes working with frontend code a lot more fun.

We’ve had a yarn hot command for a while but it didn’t support Hot Module Replacement - all it did was use the Webpack Dev Server for assets - JS, CSS, images, etc.

The first hurdle was keeping the generated files completely in memory. Since our asset pipeline was being handled by Rails, the Webpack bundles were being written to the disk. Switching to in-memory mode for Webpack meant that these files could only be accessed by visiting localhost:8080(the default). Since Rails served the files present on disk, this was a problem.

I solved this by setting up a proxy using rack-proxy. Every request to /assets for forwarded to the Webpack Dev Server running on port 8080.

Once that was done, adding HMR was surprisingly easy. I imagined adding it would require some sort of Rails plugin like webpacker and was skeptical about all this. But it turned out to be not that complicated. I had some issues with TinyMCE, but those were resolved by importing TinyMCE directly, instead of appending a script tag to the body manually.

Here’s how it all looked in the end!

What’s Next

This obviously will change(a lot of the things I listed today weren’t part of my original proposal but came up during discussions with my mentors) but here are things that I would like to complete by the time I write my next report -

1. Replacing JSONP JQuery requests with fetch
2. Converting a handful of server-rendered lists to React components
3. Removing unsafe component methods from React components

And that’s about it for now! There’s still a lot of work to do, but I’m rather proud of everything that we’ve done so far.

Hope to see you here again in 2 weeks!