Have you ever came across a dreadful and enigmatic error message stating that something like `__WEBPACK_IMPORTED_MODULE_8__node_details__[“a” /* default */]` is undefined? I have a few times already…
It is caused by circular dependencies that cannot be resolved synchronously by webpack. One is undefined, hence the error. Let me explain what it means, how it can be tackled and resolved once and for all.
Before we dive into this topic let me explain the context and environment used for the investigation. The problem arose in a JS application built with webpack 3.8 using babel and ES6 imports. It is under heavy refactor from jQuery to be fully based on React and Redux.
Circular dependencies
If you are not familiar with the term, let me briefly explain what that is and why would you prefer to avoid that in your code.
According to Wikipedia, In software engineering, a circular dependency is a relation between two or more modules which either directly or indirectly depend on each other to function properly.
They are not always evil, but you might want to treat them with special care. They cause tight coupling of the mutually dependent modules. These kinds of modules are harder to understand and reuse as doing so might cause a ripple effect where a local change to one module has global effects. It might indicate lack of a larger context or proper architecture. A good architecture imposes uni-directional flow between modules and entire layers. Top-level modules depend on lower-level modules whose entry points depend on their own internals in turn. In larger systems, though, problems sometimes come up, especially during refactoring.
On the other hand, circular dependencies are sometimes needed, i.e. in tree structures where parents refer to children and vice-versa. In functional programming it’s even encouraged to have recursive definitions to some extent. Nonetheless I would strongly discourage you from intentionally introducing circular dependencies in JavaScript.
ES6 Circular dependencies
In ES6 modules can have a single default export as well as many more granular named ones. Each module can be thought of as a single file. One of the key design goals of ES6 modules was support for circular dependencies. There were a few reasons for that. The main reason is not to break the module system while refactoring and let software engineers decide if it’s needed. Webpack supports that nicely. It’s worth noting that using const over function while defining functions prevents function hoisting within a single module and ensures the absence of circular dependencies within a single module.
What causes the problem?
The synchronous cycle of operations on imported values and/or function calls appear to be problematic. There are two symptoms I’ve run into:
Imported value is undefined when it belongs to a cycle – it happens for both expressions and function definitions.
It might appear that the import is undefined, but can also cause unexpected behavior due to the way undefined is handled in JS, i.e. there might be NaN instead of a number.
In our case it happened in unit tests first as the dependency tree is smaller and entry point is different than in the browser.
Bear in mind that cycles can be of any size. The smallest cycle consists of two modules. Here are some simplified examples to make it easier to understand:.
Pic. 1. Synchronous cycle between A and B
// A.js import B from './B'; export default 3 + B; // B.js import A from './A'; export default 4 + A; // index.js import A from './A'; console.log(A); // NaN (adding number to undefined)
// A.js
import B from './B';
export default class A extends B {};
// B.js
import A from './A';
export default class B extends A {};
// index.js
import A from './A'; // TypeError: Super expression must either be null or a function, not undefined
RangeError: exceeding call stack – it happens when imports form a cycle of immediate synchronous function calls.
// A.js import B from './B'; export default () => 3 + B(); // B.js import A from './A'; export default () => 4 + A(); // index.js import A from './A'; A(); // RangeError: Maximum call stack size exceeded
What doesn’t cause the problem?
Function calls don’t cause the problem when cycle is asynchronous meaning that directly referenced functions are not called immediately.
Pic. 2. Partially asynchronous cycle between A and B
Two cases would be:
Cycle of function calls when one continues chain through a DOM event listener being async, i.e. waiting for user click.
// A.js
import B from './B';
export default () => {
console.log('A called');
document.addEventListener('click', B, false);
};
// B.js
import A from './A';
export default () => {
console.log('B called');
A();
};
// index.js
import A from './A';
A(); // right away : A called, after click : B called, A called
Referring one class from another as long as the dependency is not necessary for immediate evaluation of class prototype – otherwise it fails as shown previously.
// A.js
import B from './B';
export default class A {
static getB() {
return new B();
}
};
// B.js
import A from './A';
export default class B {
constructor() {
this.a = new A();
}
};
// index.js
import A from './A';
console.log(A.getB().a); // instance of A
Dependency analysis
Let’s find out where the circular dependencies are. For the analysis entire modules (files) are treated as indivisible units. Dependencies in a project can be represented as a directed graph where the beginning of an edge is the importing module (file where import is used) and an end of it is the exporting module (file with export). According to WolframAlpha, A cyclic graph is a graph containing at least one graph cycle. A graph that is not cyclic is said to be acyclic (…) Cyclic graphs are not trees. Dependency graph should be acyclic. Hence the term dependency tree. Our goal is to identify cycles in the graph.
Extracting information about imports from code
ES6 Module statements can be read and statically analyzed with analyze-es6-modules package.
Let’s start by adding the package to the project as a development dependency.
yarn add analyze-es6-modules -D
The next step includes necessary configuration which in our case is as follows:
const analyzeModules = require('analyze-es6-modules');
const configuration = {
cwd: 'app/assets/javascripts',
sources: ['**/*.js'],
babel: {
plugins: [
require('babel-plugin-syntax-jsx'),
require('babel-plugin-syntax-flow'),
require('babel-plugin-syntax-object-rest-spread'),
],
},
};
const resolvedHandler = ({ modules }) => {
// do something with extracted modules
};
const rejectedHandler = () => {
console.log('rejected!');
};
analyzeModules(configuration).then(resolvedHandler, rejectedHandler);
You might need to adjust the config as:
cwd is path to a directory holding all of JS sources (without node_modules)sources is a list of globbing patterns pointing to source filesbabel.plugins is a list of babel plugins necessary to parse the files as non-standard syntax might be used. Safe option is to list all the packages that start with babel-plugin-syntax. The list can be obtained from yarn.lock.
egrep '^babel-plugin-syntax' yarn.lock
The next step is to build an intermediary representation in which each import statement gets represented as an ordered pair. The set is used as it’s a container holding only unique items.
const collectDependencies = (modules) => {
const dependencySet = new Set();
const separator = ',';
modules.forEach(({ path, imports }) => {
const importingPath = path;
imports.forEach(({ exportingModule }) => {
const exportingPath = exportingModule.resolved;
const dependency = [importingPath, exportingPath].join(separator);
dependencySet.add(dependency);
});
});
return Array.from(dependencySet.values()).map(it => it.split(separator));
};
Building directed graph
Having collected all the unique ordered pairs, they can be thought of as edges of a directed graph. Hence all that is needed is to group edges by source node and the graph represented as a hash map will be ready.
const buildDirectedGraphFromEdges = (edges) => {
return edges.reduce((graph, [sourceNode, targetNode]) => {
graph[sourceNode] = graph[sourceNode] || new Set();
graph[sourceNode].add(targetNode);
return graph;
}, {});
};
Minimizing graph to contain only cycles
At this point graph represents potentially all the module statements in the project. Most of them are not needed as they are not participating to any cycle. Hence can be removed from the graph as they are irrelevant to our analysis. Such edges share common characteristic — they are terminal on at least one end. When a module is not imported anywhere, it is a terminal node at the beginning of an edge. A terminal end node is a module that does not import anything.
const without = (firstSet, secondSet) => (
new Set(Array.from(firstSet).filter(it => !secondSet.has(it)))
);
const mergeSets = (sets) => {
const sumSet = new Set();
sets.forEach((set) => {
Array.from(set.values()).forEach((value) => {
sumSet.add(value);
});
});
return sumSet;
};
const stripTerminalNodes = (graph) => {
const allSources = new Set(Object.keys(graph));
const allTargets = mergeSets(Object.values(graph));
const terminalSources = without(allSources, allTargets);
const terminalTargets = without(allTargets, allSources);
const newGraph = Object.entries(graph).reduce((smallerGraph, [source, targets]) => {
if (!terminalSources.has(source)) {
const nonTerminalTargets = without(targets, terminalTargets);
if (nonTerminalTargets.size > 0) {
smallerGraph[source] = nonTerminalTargets;
}
}
return smallerGraph;
}, {});
return newGraph;
};
The process of stripping terminal nodes can be repeated as long as the graph gets smaller.
const calculateGraphSize = (graph) => mergeSets(Object.values(graph)).size;
const miminizeGraph = (graph) => {
const smallerGraph = stripTerminalNodes(graph);
if (calculateGraphSize(smallerGraph) < calculateGraphSize(graph)) {
return miminizeGraph(smallerGraph);
} else {
return smallerGraph;
}
};
Graph visualization with DOT
At this point, the minimal graph has been identified, but it would be helpful to visualize it as it might be difficult to understand the cycles from a plain object representation. Simple way to do so is to represent the graph using dot language and render with graphviz.
const convertDirectedGraphToDot = (graph) => {
const stringBuilder = [];
stringBuilder.push('digraph G {');
Object.entries(graph).forEach(([source, targetSet]) => {
const targets = Array.from(targetSet);
stringBuilder.push('"' + source + '" -> { "' + targets.join('" "') + '" }');
});
stringBuilder.push('}');
return stringBuilder.join("\n");
};
Finally, the resolvedHandler is completed and looks as follows:
const resolvedHandler = ({ modules }) => {
const dependencies = collectDependencies(modules);
const graph = buildDirectedGraphFromEdges(dependencies);
const minimalGraph = miminizeGraph(graph);
console.log(convertDirectedGraphToDot(minimalGraph));
};
Printed text can be copied to an online renderer. In our case the graph we got looked as below.
Pic. 3. Visualization of the minimal graph using webgraphviz.com
This type of visual representation is very easy for humans to interpret. There are very short cycles, i.e. node_details <-> components/NodeSidebar and longer ones, but main attractors remain node_details and node_details_config. As a rule of thumb all edges should point in the same direction, i.e. from top to bottom or from top-left to bottom-right. Any edge that goes the other way is worth looking into. The Graph will be empty when there are no cycles.
The entire script with instruction on how to run it using node is available as a gist.
Alternative ways
If you are not into images, you can use circular-dependency-plugin for webpack which prints a list of cycles during compilation of assets. Minimal change is needed in webpack.config.js – just add the plugin to the list of plugins.
const CircularDependencyPlugin = require('circular-dependency-plugin');
// webpack config
const config = {
// … omitted for brevity
plugins: [
// … other plugins
new CircularDependencyPlugin({
// exclude detection of files based on a RegExp
exclude: /a\.js|node_modules/,
// add errors to webpack instead of warnings
failOnError: false,
// set the current working directory for displaying module paths
cwd: process.cwd(),
})
]
};
It will produce output like the following.
WARNING in Circular dependency detected: app/assets/javascripts/process_allocations.js -> app/assets/javascripts/processes.js -> app/assets/javascripts/process_allocations.js
Another option is to use additional rules for eslint – dependencies/no-cycles rule from eslint-plugin-dependencies or import/no-cycle rule from eslint-plugin-import. Both do the job, but former is significantly faster for longer cycles and provides output that is easier to analyse – very similar to the one above from webpack plugin.
Resolving problematic dependencies
Having found all the issues, let’s talk about ways to address them. Solutions may vary depending on the problem. We have used the following strategies.
Move problematic exports to another file preferably terminal one which can be imported from all other places without causing cycles.
Introduce event triggers and handlers to loose tight coupling.
Invert control by injecting dependencies.
Since it might be vague, let me give you a contrived example of the last strategy. Consider the following example.
// A.js
import B from './B';
export default class A {
foo() {
return 'A:foo:' + this.getB().bar();
}
bar() {
return 'A:bar';
}
getB() {
if (!this.b) this.b = new B();
return this.b;
}
};
// B.js
import A from './A';
export default class B {
foo() {
return 'B:foo:' + this.getA().bar();
}
bar() {
return 'B:bar';
}
getA() {
if (!this.a) this.a = new A();
return this.a;
}
};
// index.js
import A from './A';
import B from './B';
console.log(new A().foo() + new B().foo()); // A:foo:B:barB:foo:A:bar
Circular dependency can be removed if one of the classes is injected via constructor as shown below.
// A.js
import B from './B';
export default class A {
foo() {
return 'A:foo:' + this.getB().bar();
}
bar() {
return 'A:bar';
}
getB() {
if (!this.b) this.b = new B(this);
return this.b;
}
};
// B.js
export default class B {
constructor(a) { // the main change
this.a = a;
}
foo() {
return 'B:foo:' + this.getA().bar();
}
bar() {
return 'B:bar';
}
getA() {
return this.a;
}
};
// index.js
import A from './A';
import B from './B';
const a = new A();
const b = new B(a);
console.log(a.foo() + b.foo()); // A:foo:A:barB:foo:A:bar
After doing so, you can use one of the methods described above to verify if it helped.
Monitoring dependencies
When so much effort is put into a problem, it’s a good idea to proactively monitor and prevent it from happening again. We have decided to use circular-dependency-plugin for webpack. It’s configured in a way that fails on error – failOnError: true. We use that to get an immediate error on CI while compiling assets for feature specs. Refer to advanced usage section in its README if you need to whitelist some existing cycles. If you prefer to run tests nonetheless, then eslint seems like a better option.
Summary
It has been quite a journey :) Congrats you’ve reached the end. Hopefully you have never encountered such a problem, but if you are here, I guess you already have :) In that case I hope you will find the described approaches helpful and will not stumble across the dreadful error message about missing dependencies ever again. If you have scrolled down here for a quick tip – you will find the entire script with all the instructions here will give you the gist.