diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 911d74c86..1ab9f13ac 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -11,6 +11,7 @@
- [Output Files](file_formats/output_files.md)
- [Model Description](model/README.md)
- [Units and Dimensions](model/units_and_dimensions.md)
+ - [Commodity Graphs](model/commodity_graphs.md)
- [Dispatch Optimisation](model/dispatch_optimisation.md)
- [Investment Appraisal](model/investment.md)
- [Commodity Prices](model/prices.md)
diff --git a/docs/glossary.md b/docs/glossary.md
index fad6db724..852f33f2c 100644
--- a/docs/glossary.md
+++ b/docs/glossary.md
@@ -82,6 +82,10 @@ electricity.
produced and/or consumed by *Processes* in the model. A *Service Demand* is a type of *Commodity* that
is defined at the end point of the system.
+**Commodity Graph:** A directed graph built by MUSE2 for each *Milestone Year*, in which nodes are
+*Commodity Markets* and edges are *Processes*. It is used to validate the model, determine the
+*Investment Order* and identify *Circular Commodity Dependencies*.
+
**Commodity Levy:** Represents a tax, levy or other external cost on a *Commodity*. Levies can be
applied to all *Commodity* production (sum of output of all *Assets* for that *Commodity*), net
production (sum of output and input for all *Assets*), or all consumption (sum of input for all
diff --git a/docs/images/commodity_graph_example.dot b/docs/images/commodity_graph_example.dot
new file mode 100644
index 000000000..9b5288f9d
--- /dev/null
+++ b/docs/images/commodity_graph_example.dot
@@ -0,0 +1,41 @@
+// Example commodity graph for docs/model/commodity_graphs.md
+// Render with: dot -Tsvg commodity_graph_example.dot -o commodity_graph_example.svg
+digraph commodity_graph {
+ rankdir=LR;
+ nodesep=0.4;
+ ranksep=0.8;
+ node [shape=box, style="rounded,filled", fontname="Helvetica", fontsize=12];
+ edge [fontname="Helvetica", fontsize=10];
+
+ SOURCE [shape=ellipse, style=dashed];
+
+ GASPRD [label="GASPRD\n(gas produced)", fillcolor="#B3CDE3"];
+ GASNAT [label="GASNAT\n(natural gas)", fillcolor="#B3CDE3"];
+ ELCTRI [label="ELCTRI\n(electricity)", fillcolor="#B3CDE3"];
+ H2YPRD [label="H2YPRD\n(hydrogen)", fillcolor="#B3CDE3"];
+ RSHEAT [label="RSHEAT\n(residential heat)", fillcolor="#CCEBC5"];
+ TPASKM [label="TPASKM\n(passenger km)", fillcolor="#CCEBC5"];
+ CO2EMT [label="CO2EMT\n(CO₂ emitted)", fillcolor="#FED9A6"];
+
+ SOURCE -> GASPRD [label="GASDRV"];
+ SOURCE -> ELCTRI [label="WNDFRM"];
+ GASPRD -> GASNAT [label="GASPRC"];
+ GASNAT -> ELCTRI [label="GASCGT"];
+ GASNAT -> CO2EMT [label="GASCGT", style=dashed];
+ GASNAT -> RSHEAT [label="RGASBR"];
+ GASNAT -> CO2EMT [label="RGASBR", style=dashed];
+ ELCTRI -> H2YPRD [label="H2YPRO"];
+ H2YPRD -> ELCTRI [label="H2YGEN"];
+ ELCTRI -> RSHEAT [label="RELCHP"];
+ ELCTRI -> TPASKM [label="TELCCR"];
+
+ legend [shape=plain, style="", fontsize=11, label=<
+
+
Key
+
SVD
service demand
+
SED
supply equals demand
+
OTH
other
+
───
primary output
+
- - -
secondary output
+
>];
+}
diff --git a/docs/images/commodity_graph_example.svg b/docs/images/commodity_graph_example.svg
new file mode 100644
index 000000000..9c34a6836
--- /dev/null
+++ b/docs/images/commodity_graph_example.svg
@@ -0,0 +1,174 @@
+
+
+
+
+
diff --git a/docs/images/investment_order_example.dot b/docs/images/investment_order_example.dot
new file mode 100644
index 000000000..d335fdf31
--- /dev/null
+++ b/docs/images/investment_order_example.dot
@@ -0,0 +1,36 @@
+// Investment order example for docs/model/commodity_graphs.md
+// Render with: dot -Tsvg investment_order_example.dot -o investment_order_example.svg
+digraph investment_order {
+ rankdir=LR;
+ nodesep=0.4;
+ ranksep=0.8;
+ node [shape=box, style="rounded,filled", fontname="Helvetica", fontsize=12];
+ edge [fontname="Helvetica", fontsize=10];
+ fontname="Helvetica";
+ fontsize=12;
+
+ GASPRD [label="GASPRD\n(invested 4th)", fillcolor="#B3CDE3"];
+ GASNAT [label="GASNAT\n(invested 3rd)", fillcolor="#B3CDE3"];
+
+ subgraph cluster_cycle {
+ label="Circularity (invested 2nd)";
+ style="rounded,dashed";
+ ELCTRI [label="ELCTRI\n(2a)", fillcolor="#B3CDE3"];
+ H2YPRD [label="H2YPRD\n(2b)", fillcolor="#B3CDE3"];
+ }
+
+ subgraph cluster_layer {
+ label="Layer (invested 1st)";
+ style="rounded,dashed";
+ TPASKM [fillcolor="#CCEBC5"];
+ RSHEAT [fillcolor="#CCEBC5"];
+ }
+
+ GASPRD -> GASNAT;
+ GASNAT -> ELCTRI;
+ GASNAT -> RSHEAT;
+ ELCTRI -> H2YPRD;
+ H2YPRD -> ELCTRI;
+ ELCTRI -> RSHEAT;
+ ELCTRI -> TPASKM;
+}
diff --git a/docs/images/investment_order_example.svg b/docs/images/investment_order_example.svg
new file mode 100644
index 000000000..b5246f605
--- /dev/null
+++ b/docs/images/investment_order_example.svg
@@ -0,0 +1,105 @@
+
+
+
+
+
diff --git a/docs/images/save_graphs_simple.dot b/docs/images/save_graphs_simple.dot
new file mode 100644
index 000000000..aa492cd08
--- /dev/null
+++ b/docs/images/save_graphs_simple.dot
@@ -0,0 +1,22 @@
+// Unmodified output of `muse2 save-graphs examples/simple` (2020.dot)
+// Used in docs/model/commodity_graphs.md
+// Render with: dot -Tsvg save_graphs_simple.dot -o save_graphs_simple.svg
+digraph {
+ 0 [ label = "SOURCE" ]
+ 1 [ label = "GASPRD | GBR" ]
+ 2 [ label = "CO2EMT | GBR" ]
+ 3 [ label = "GASNAT | GBR" ]
+ 4 [ label = "ELCTRI | GBR" ]
+ 5 [ label = "RSHEAT | GBR" ]
+ 0 -> 1 [ label = "GASDRV | GBR" ]
+ 0 -> 2 [ label = "GASDRV | GBR" style=dashed]
+ 1 -> 3 [ label = "GASPRC | GBR" ]
+ 1 -> 2 [ label = "GASPRC | GBR" style=dashed]
+ 0 -> 4 [ label = "WNDFRM | GBR" ]
+ 3 -> 4 [ label = "GASCGT | GBR" ]
+ 3 -> 2 [ label = "GASCGT | GBR" style=dashed]
+ 3 -> 5 [ label = "RGASBR | GBR" ]
+ 3 -> 2 [ label = "RGASBR | GBR" style=dashed]
+ 4 -> 5 [ label = "RELCHP | GBR" ]
+ 0 -> 4 [ label = "SLRFRM | GBR" ]
+}
diff --git a/docs/images/save_graphs_simple.svg b/docs/images/save_graphs_simple.svg
new file mode 100644
index 000000000..f80ed112f
--- /dev/null
+++ b/docs/images/save_graphs_simple.svg
@@ -0,0 +1,125 @@
+
+
+
+
+
diff --git a/docs/model/README.md b/docs/model/README.md
index 01dcf8c9c..d6ec1c795 100644
--- a/docs/model/README.md
+++ b/docs/model/README.md
@@ -62,9 +62,9 @@ Assets whose scheduled decommissioning year has been reached are removed from th
### 2. Agent investment
Agents select assets to meet commodity demand for the current MSY. Investment decisions follow a
-pre-computed **investment order** derived from the topology of the commodity graph: markets are
-processed deepest-first (most downstream to most upstream), grouped into **layers** of independent
-markets at the same depth. See [Investment Appraisal][investment] for full details.
+pre-computed **investment order** derived from the topology of the [commodity graph][commodity-graphs]:
+markets are processed deepest-first (most downstream to most upstream), grouped into **layers** of
+independent markets at the same depth. See [Investment Appraisal][investment] for full details.
After all markets in each layer are settled, a **partial system dispatch** is run over all assets
selected so far. This propagates demand upstream: input commodity flows consumed by newly committed
@@ -91,6 +91,7 @@ resulting flows and commodity prices are written to the output files and carried
price basis for the next MSY's investment appraisal.
[investment]: ./investment.md
+[commodity-graphs]: ./commodity_graphs.md
[dispatch-optimisation]: ./dispatch_optimisation.md
[prices]: ./prices.md
[model-toml]: ../file_formats/input_files.md#model-parameters-modeltoml
diff --git a/docs/model/commodity_graphs.md b/docs/model/commodity_graphs.md
new file mode 100644
index 000000000..c28c0b9c9
--- /dev/null
+++ b/docs/model/commodity_graphs.md
@@ -0,0 +1,228 @@
+# Commodity Graphs
+
+When MUSE2 loads a model, it uses your commodity, process and process flow input files to build a
+**commodity graph** for each milestone year. You never write these graphs yourself, but they
+underpin how MUSE2 checks and runs your model. Understanding them will help you to build models
+that work, and to work out what has gone wrong when they don't.
+
+## Why MUSE2 builds commodity graphs
+
+- **To map out how commodities can flow.** The graph records which commodities each process can
+ turn into which other commodities, in each region. In other words, it captures the structure of
+ your energy system.
+- **To validate your model.** MUSE2 checks that every service demand can be met by the available
+ processes, and that every commodity obeys the rules for its type (see
+ [Validating the graph](#validating-the-graph)).
+- **To protect the integrity of the system.** Structural mistakes, such as a missing process or a
+ commodity with the wrong type, are caught with a clear error message when the model is loaded,
+ rather than causing confusing failures partway through a run.
+- **To decide the order of investment and price calculations.** MUSE2 invests from downstream to
+ upstream (e.g. in residential heating before electricity, and in electricity before gas) and
+ calculates prices from upstream to downstream (see
+ [Investment and price order](#investment-and-price-order)).
+- **To identify circularities**, where commodities depend on one another in a loop (e.g.
+ electricity is used to make hydrogen, which is used to generate electricity). These need special
+ treatment during [investment][investment-circularities] and [pricing][pricing-circularities].
+- **To help you visualise your model**, via the `muse2 save-graphs` command (see
+ [Visualising your model](#visualising-your-model)).
+- **For performance.** The graph shows which markets are independent of one another. MUSE2 already
+ uses this to group independent markets together, and in future it could allow independent parts
+ of a model to be solved in parallel.
+
+## What's in a commodity graph
+
+- Each **node** is a [commodity market][glossary]: one commodity in one region (e.g. electricity in
+ GBR).
+- Each **edge** is a process, drawn as an arrow from an input commodity to an output commodity. A
+ process with several inputs or outputs is drawn as several edges, one for each input/output pair.
+- Processes with no inputs (e.g. wind farms or gas extraction) are drawn from a special `SOURCE`
+ node. Processes with no outputs are drawn to a special `SINK` node.
+- Edges are either **primary** or **secondary**:
+ - a **primary** edge leads to the process's primary output (the `primary_output` column in
+ `processes.csv`). This is the market in which agents can invest in the process.
+ - a **secondary** edge leads to any other output, such as CO₂ emitted as a by-product.
+
+MUSE2 builds a separate graph for each milestone year, as the available processes can change over
+time. A process is included in a year's graph if an asset of that process *could* be operating in
+that year, taking into account the process's start and end years and its lifetime.
+
+### Example
+
+Consider a single-region model with the following processes. This is a simplified version of the
+`circularity` [example model][example models].
+
+| Process | Description | Inputs | Outputs (primary output in bold) |
+| ------- | --------------------- | ------ | -------------------------------- |
+| GASDRV | Gas extraction | – | **GASPRD** |
+| GASPRC | Gas processing | GASPRD | **GASNAT** |
+| WNDFRM | Wind farm | – | **ELCTRI** |
+| GASCGT | Gas turbine | GASNAT | **ELCTRI**, CO2EMT |
+| H2YPRO | Hydrogen electrolyser | ELCTRI | **H2YPRD** |
+| H2YGEN | Hydrogen turbine | H2YPRD | **ELCTRI** |
+| RGASBR | Gas boiler | GASNAT | **RSHEAT**, CO2EMT |
+| RELCHP | Heat pump | ELCTRI | **RSHEAT** |
+| TELCCR | Electric car | ELCTRI | **TPASKM** |
+
+Figure 1 shows the commodity graph for this model, with nodes coloured by commodity type.
+
+
+
+*Figure 1: Commodity graph for the example model.*
+
+Note that the gas turbine (GASCGT) appears as two edges: a solid edge to its primary output
+(electricity) and a dashed edge to CO₂, which it emits as a by-product. Electricity and hydrogen
+form a **circularity**, as electricity is used to produce hydrogen (H2YPRO), which can be used to
+generate electricity (H2YGEN).
+
+## Validating the graph
+
+When you run `muse2 validate` or `muse2 run`, MUSE2 checks each year's graph against the rules for
+each commodity type. As process availabilities and service demands can vary through the year, the
+checks are repeated for each part of the year, at the level given by the commodity's
+`time_slice_level` (in `commodities.csv`):
+
+- `annual` commodities are checked once for the whole year;
+- `season` commodities are checked for each season;
+- `daynight` commodities are checked for each individual time slice.
+
+For each check, any process that cannot operate during that period (i.e. its availability in
+`process_availabilities.csv` is zero throughout the period) is removed from the graph, and the
+service demand for that period is taken into account. The rules are:
+
+| Commodity type | Rule |
+| --- | --- |
+| `svd` | Cannot be an input to any process; if demanded, it must have a producer. |
+| `sed` | If any process can consume it, a process must be able to produce it. |
+| `oth` | Can be produced or consumed, but not both. |
+
+### Diagnosing validation errors
+
+If one of these rules is broken, MUSE2 will tell you which commodity, region, year and period
+caused the problem. For example, if the electrolyser (H2YPRO) were removed from the example model
+above, you would see:
+
+```txt
+Caused by:
+ 0: Error validating commodity graph for 2020 in winter
+ 1: SED commodity H2YPRD in region GBR may be consumed but has no producers
+```
+
+Hydrogen can still be consumed by the hydrogen turbine, but there is nothing left to produce it.
+The problem was found in the season `winter`, as hydrogen has a `season` time slice level. MUSE2
+stops at the first problem it finds, so the same problem may affect other seasons too.
+
+Some tips for fixing validation errors:
+
+- **Look at the graph** for the year in the error message using
+ [`muse2 save-graphs`](#visualising-your-model), which works even when validation fails.
+- **If the error only occurs in some periods, check process availabilities.** For example, solar
+ alone cannot supply a `daynight` commodity at night.
+- **If the error only occurs in some years, check process start/end years and lifetimes.**
+- **Check commodity types.** For example, CO₂ that is both emitted and captured for storage must be
+ `sed` rather than `oth`.
+
+Note that validation only checks the structure of the model. It does not check whether there is
+enough capacity to meet demand, whether costs are sensible, or whether agents are able to invest in
+the processes they need, so a model that passes validation can still fail when it is run.
+
+## Investment and price order
+
+In each milestone year, MUSE2 invests in one market (or group of markets) at a time, from the most
+downstream to the most upstream, so that demand from assets already selected downstream (e.g. heat
+pumps consuming electricity) is known when upstream markets are invested in (see
+[Investment Appraisal][investment]). Prices are calculated in the reverse order, so that the prices
+of a process's inputs are known before the price of its output is calculated (see
+[Commodity Prices][prices]).
+
+The investment order is worked out from the commodity graph for each year as follows:
+
+1. **Keep only the markets that are invested in.** Only `svd` and `sed` markets are kept. `oth`
+ commodities and the `SOURCE` and `SINK` nodes are ignored. Both primary and secondary edges are
+ kept.
+2. **Group circularities.** Each set of markets that form a loop is treated as a single group (in
+ graph theory terms, the strongly connected components of the graph are collapsed into single
+ nodes). The graph now has no loops.
+3. **Sort the markets.** The markets are put in order so that every market comes before the
+ markets that supply it (a [topological sort]).
+4. **Group independent markets into layers.** Each market is given a depth, which is the length of
+ the longest supply chain leading to it. Markets with the same depth do not depend on one another,
+ so they are grouped into a **layer**. MUSE2 invests in all markets in a layer before running a
+ single [partial system dispatch][partial-dispatch], rather than one per market, which saves time.
+
+Markets within a circularity cannot be put into an order that respects every dependency, as each
+market in the loop depends on the others. MUSE2 chooses an order for them by solving a small
+optimisation problem; see the [API documentation for `order_sccs`][order-sccs] for details. For how
+investment then proceeds within a circularity, see [Circularities][investment-circularities].
+
+### Example investment order
+
+Figure 2 shows the investment order for the [example model](#example). CO2EMT (an `oth` commodity)
+and the `SOURCE` node are ignored, and electricity and hydrogen are grouped into a circularity.
+Residential heat and passenger kilometres have no dependencies between them, so they form a layer.
+
+
+
+*Figure 2: Investment order for the example model. Investment proceeds from right to left and
+prices are calculated in the reverse order, from left to right.*
+
+### Checking the investment order for your model
+
+If you set the [log level] to `debug`, MUSE2 will print the investment order for each milestone
+year in which it makes investments. For the example model, this looks like:
+
+
+
+```txt
+Investment order for year '2030': [TPASKM|GBR, RSHEAT|GBR] -> (ELCTRI|GBR, H2YPRD|GBR) -> GASNAT|GBR -> GASPRD|GBR
+```
+
+
+
+Each market is written as `COMMODITY|REGION`. Square brackets (`[...]`) show a layer and round
+brackets (`(...)`) show a circularity. Markets within a layer are independent, so their order within
+the layer doesn't matter.
+
+## Visualising your model
+
+To draw the commodity graphs for your model, run:
+
+```sh
+muse2 save-graphs path/to/model
+```
+
+This writes one file per milestone year (e.g. `2020.dot`) to `muse2_graphs//`. You
+can choose a different folder with the `--output-dir` option (see the
+[command line help][save-graphs]). The files are in [DOT format], which you can draw with
+[Graphviz] (e.g. `dot -Tsvg 2020.dot -o 2020.svg`) or by pasting them into [Graphviz Online].
+
+Figure 3 shows the output for the `simple` example in 2020, which you can reproduce by running
+`muse2 example extract simple` followed by `muse2 save-graphs simple`.
+
+
+
+*Figure 3: Output of `muse2 save-graphs` for the `simple` example in 2020.*
+
+When reading these graphs, note that:
+
+- nodes are labelled `COMMODITY | REGION` and edges are labelled `PROCESS | REGION`;
+- dashed edges are secondary outputs;
+- nodes are not coloured by commodity type, so check `commodities.csv` to see which commodities are
+ `svd`, `sed` or `oth`;
+- the graph shows every process that could operate in that year. It does not take into account
+ process availabilities or service demands, which are only considered during
+ [validation](#validating-the-graph).
+
+[glossary]: ../glossary.md
+[example models]: ../examples.md
+[investment]: ./investment.md
+[prices]: ./prices.md
+[topological sort]: https://en.wikipedia.org/wiki/Topological_sorting
+[partial-dispatch]: ./investment.md#partial-system-dispatch
+[order-sccs]: ../api/muse2/graph/investment/fn.order_sccs.html
+[investment-circularities]: ./investment.md#circularities
+[pricing-circularities]: ./prices.md#circularities
+[log level]: ../user_guide.md#setting-the-log-level
+[save-graphs]: ../command_line_help.md#muse2-save-graphs
+[DOT format]: https://graphviz.org/doc/info/lang.html
+[Graphviz]: https://graphviz.org/
+[Graphviz Online]: https://dreampuf.github.io/GraphvizOnline
diff --git a/docs/model/investment.md b/docs/model/investment.md
index a16796126..2c5e17ad4 100644
--- a/docs/model/investment.md
+++ b/docs/model/investment.md
@@ -34,6 +34,9 @@ investment decisions. Other commodity types (e.g. `OTH`) are excluded.
> Note: the investment order is the reverse of the [price calculation order][prices], where prices
> are computed upstream first.
+For details of how the investment order is determined from the commodity graph, including a worked
+example, see [Commodity Graphs](./commodity_graphs.md#investment-and-price-order).
+
### Circularities
When commodity markets form a cycle (e.g. electricity → hydrogen → electricity), the markets in
diff --git a/docs/model/prices.md b/docs/model/prices.md
index d03bf689a..f50930b10 100644
--- a/docs/model/prices.md
+++ b/docs/model/prices.md
@@ -111,7 +111,8 @@ downstream to intermediate energy carriers and final service demands.
This sequence is the **reverse** of the investment order. This ensures that the prices of input
commodities (from upstream markets) are calculated and known before they are consumed as inputs by
-downstream processes evaluating marginal or full cost pricing.
+downstream processes evaluating marginal or full cost pricing. For details of how this order is
+determined, see [Commodity Graphs](./commodity_graphs.md#investment-and-price-order).
## Circularities
diff --git a/docs/user_guide.md b/docs/user_guide.md
index dd5c08692..03ffee9c1 100644
--- a/docs/user_guide.md
+++ b/docs/user_guide.md
@@ -125,10 +125,14 @@ with [Graphviz online].
Dashed lines are used to indicate flows for non-primary outputs of a process (as defined in the
`processes.csv` input file).
+For more information on commodity graphs, including how to read them and how MUSE2 uses them to
+validate your model and decide the order of investments, see [Commodity Graphs].
+
[`muse2 save-graphs` command]: command_line_help.md#muse2-save-graphs
[DOT format]: https://graphviz.org/doc/info/lang.html
[Graphviz]: https://graphviz.org/
[Graphviz online]: https://dreampuf.github.io/GraphvizOnline
+[Commodity Graphs]: model/commodity_graphs.md
## Modifying the program settings
diff --git a/schemas/input/commodities.yaml b/schemas/input/commodities.yaml
index 25e76b8c2..a55c2b1a7 100644
--- a/schemas/input/commodities.yaml
+++ b/schemas/input/commodities.yaml
@@ -3,8 +3,8 @@ description: |
Describes commodities in the system.
notes:
- - Every SED (supply equals demand) commodity must have both producer and consumer processes for
- every region and milestone year.
+ - Every SED (supply equals demand) commodity that can be consumed by a process must also have a
+ producer process, for every region and milestone year.
- Every SVD (service demand) commodity must have a producer for every region and milestone year.
fields: