Analyzing circular dependencies in ES6

How to Analyze Circular Dependencies in ES6?

Have you ever came across a dreadful and enigmatic error message stating that something like __WEBPACK_IMPORTED_MODULE_8__node_details__["a" /* default */] is undefined? I have a few times already…

It is caused by circular dependencies that cannot be resolved synchronously by webpack. One is undefined, hence the error. Let me explain what it means, how it can be tackled and resolved once and for all.

Before we dive into this topic let me explain the context and environment used for the investigation. The problem arose in a JS application built with webpack 3.8 using babel and ES6 imports. It is under heavy refactor from jQuery to be fully based on React and Redux.

Circular dependencies in ES6

Circular dependencies

If you are not familiar with the term, let me briefly explain what that is and why would you prefer to avoid that in your code.

According to Wikipedia, In software engineering, a circular dependency is a relation between two or more modules which either directly or indirectly depend on each other to function properly.

They are not always evil, but you might want to treat them with special care. They cause tight coupling of the mutually dependent modules. These kinds of modules are harder to understand and reuse as doing so might cause a ripple effect where a local change to one module has global effects. It might indicate lack of a larger context or proper architecture. A good architecture imposes uni-directional flow between modules and entire layers. Top-level modules depend on lower-level modules whose entry points depend on their own internals in turn. In larger systems, though, problems sometimes come up, especially during refactoring.

On the other hand, circular dependencies are sometimes needed, i.e. in tree structures where parents refer to children and vice-versa. In functional programming it’s even encouraged to have recursive definitions to some extent. Nonetheless I would strongly discourage you from intentionally introducing circular dependencies in JavaScript.

ES6 Circular dependencies

In ES6 modules can have a single default export as well as many more granular named ones. Each module can be thought of as a single file. One of the key design goals of ES6 modules was support for circular dependencies. There were a few reasons for that. The main reason is not to break the module system while refactoring and let software engineers decide if it’s needed. Webpack supports that nicely. It’s worth noting that using const over function while defining functions prevents function hoisting within a single module and ensures the absence of circular dependencies within a single module.

What causes the problem?

The synchronous cycle of operations on imported values and/or function calls appear to be problematic. There are two symptoms I’ve run into:

Imported value is undefined when it belongs to a cycle – it happens for both expressions and function definitions. It might appear that the import is undefined, but can also cause unexpected behavior due to the way undefined is handled in JS, i.e. there might be NaN instead of a number. In our case it happened in unit tests first as the dependency tree is smaller and entry point is different than in the browser.

Bear in mind that cycles can be of any size. The smallest cycle consists of two modules. Here are some simplified examples to make it easier to understand:.

synchronous cycle between A and B Pic. 1. Synchronous cycle between A and B

RangeError: exceeding call stack – it happens when imports form a cycle of immediate synchronous function calls.

What doesn’t cause the problem?

Function calls don’t cause the problem when cycle is asynchronous meaning that directly referenced functions are not called immediately.

partially asynchronous cycle between A and B Pic. 2. Partially asynchronous cycle between A and B

Two cases would be:

Cycle of function calls when one continues chain through a DOM event listener being async, i.e. waiting for user click.

Referring one class from another as long as the dependency is not necessary for immediate evaluation of class prototype – otherwise it fails as shown previously.

Dependency analysis

Let’s find out where the circular dependencies are. For the analysis entire modules (files) are treated as indivisible units. Dependencies in a project can be represented as a directed graph where the beginning of an edge is the importing module (file where import is used) and an end of it is the exporting module (file with export). According to WolframAlpha, A cyclic graph is a graph containing at least one graph cycle. A graph that is not cyclic is said to be acyclic (…) Cyclic graphs are not trees. Dependency graph should be acyclic. Hence the term dependency tree. Our goal is to identify cycles in the graph.

Extracting information about imports from code

ES6 Module statements can be read and statically analyzed with analyze-es6-modules package.

Let’s start by adding the package to the project as a development dependency.

The next step includes necessary configuration which in our case is as follows:

You might need to adjust the config as:

cwd is path to a directory holding all of JS sources (without node_modules) sources is a list of globbing patterns pointing to source files babel.plugins is a list of babel plugins necessary to parse the files as non-standard syntax might be used. Safe option is to list all the packages that start with babel-plugin-syntax. The list can be obtained from yarn.lock.

The next step is to build an intermediary representation in which each import statement gets represented as an ordered pair. The set is used as it’s a container holding only unique items.

Building directed graph

Having collected all the unique ordered pairs, they can be thought of as edges of a directed graph. Hence all that is needed is to group edges by source node and the graph represented as a hash map will be ready.

Minimizing graph to contain only cycles

At this point graph represents potentially all the module statements in the project. Most of them are not needed as they are not participating to any cycle. Hence can be removed from the graph as they are irrelevant to our analysis. Such edges share common characteristic — they are terminal on at least one end. When a module is not imported anywhere, it is a terminal node at the beginning of an edge. A terminal end node is a module that does not import anything.

The process of stripping terminal nodes can be repeated as long as the graph gets smaller.

Graph visualization with DOT

At this point, the minimal graph has been identified, but it would be helpful to visualize it as it might be difficult to understand the cycles from a plain object representation. Simple way to do so is to represent the graph using dot language and render with graphviz.

Finally, the resolvedHandler is completed and looks as follows:

Printed text can be copied to an online renderer. In our case the graph we got looked as below.

G components/NodeAllocationHeads components/NodeAllocationHeads node_allocations node_allocations components/NodeAllocationHeads->node_allocations node_details node_details node_allocations->node_details containers/NodeAllocations containers/NodeAllocations node_allocations->containers/NodeAllocations components/NodeDetailsHeader components/NodeDetailsHeader node_details_config node_details_config components/NodeDetailsHeader->node_details_config components/NodeDetailsHeader->node_details state/selectors/node_details state/selectors/node_details components/NodeDetailsHeader->state/selectors/node_details node_filter node_filter node_details_config->node_filter processes processes node_details_config->processes process_allocations process_allocations node_details_config->process_allocations resources resources node_details_config->resources nodes nodes node_details_config->nodes node_details->components/NodeAllocationHeads node_details->components/NodeDetailsHeader node_details->node_details_config node_details->state/selectors/node_details components/NodeSections components/NodeSections node_details->components/NodeSections components/NodeSidebar components/NodeSidebar node_details->components/NodeSidebar node_details->node_filter node_details->nodes state/selectors/node_details->node_filter components/NodeSection components/NodeSection components/NodeSubsection components/NodeSubsection components/NodeSection->components/NodeSubsection components/NodeSubsection->node_details components/NodeSections->components/NodeSection components/NodeSidebar->node_details containers/EditNode containers/EditNode containers/EditNode->node_details_config state/actions/nodes state/actions/nodes containers/EditNode->state/actions/nodes state/actions/nodes->node_details_config containers/NewNode containers/NewNode containers/NewNode->node_details_config containers/NewNode->state/actions/nodes containers/NodeAllocations->node_filter node_filter->state/selectors/node_details containers/Resources containers/Resources containers/Resources->components/NodeSubsection processes->node_details processes->process_allocations process_allocations->processes resources->containers/Resources nodes->node_details nodes->containers/EditNode nodes->state/actions/nodes nodes->containers/NewNode

Pic. 3. Visualization of the minimal graph using webgraphviz.com

This type of visual representation is very easy for humans to interpret. There are very short cycles, i.e. node_details <-> components/NodeSidebar and longer ones, but main attractors remain node_details and node_details_config. As a rule of thumb all edges should point in the same direction, i.e. from top to bottom or from top-left to bottom-right. Any edge that goes the other way is worth looking into. The Graph will be empty when there are no cycles.

The entire script with instruction on how to run it using node is available as a gist.

Alternative ways

If you are not into images, you can use circular-dependency-plugin for webpack which prints a list of cycles during compilation of assets. Minimal change is needed in webpack.config.js – just add the plugin to the list of plugins.

It will produce output like the following.

Another option is to use additional rules for eslintdependencies/no-cycles rule from eslint-plugin-dependencies or import/no-cycle rule from eslint-plugin-import. Both do the job, but former is significantly faster for longer cycles and provides output that is easier to analyse – very similar to the one above from webpack plugin.

Resolving problematic dependencies

Having found all the issues, let’s talk about ways to address them. Solutions may vary depending on the problem. We have used the following strategies. Move problematic exports to another file preferably terminal one which can be imported from all other places without causing cycles. Introduce event triggers and handlers to loose tight coupling. Invert control by injecting dependencies.

Since it might be vague, let me give you a contrived example of the last strategy. Consider the following example.

Circular dependency can be removed if one of the classes is injected via constructor as shown below.

After doing so, you can use one of the methods described above to verify if it helped.

Monitoring dependencies

When so much effort is put into a problem, it’s a good idea to proactively monitor and prevent it from happening again. We have decided to use circular-dependency-plugin for webpack. It’s configured in a way that fails on error – failOnError: true. We use that to get an immediate error on CI while compiling assets for feature specs. Refer to advanced usage section in its README if you need to whitelist some existing cycles. If you prefer to run tests nonetheless, then eslint seems like a better option.

Summary

It has been quite a journey :) Congrats you’ve reached the end. Hopefully you have never encountered such a problem, but if you are here, I guess you already have :) In that case I hope you will find the described approaches helpful and will not stumble across the dreadful error message about missing dependencies ever again. If you have scrolled down here for a quick tip – you will find the entire script with all the instructions here will give you the gist.