Batch runs

Start a batch#

Build the model as usual, then press the Batch run button beside play — the fast-forward arrows. The model is checked against the wiring rules first, exactly as it is for a live run.

A batch runs as fast as your machine allows, with no animation and no live charts. The whole simulation happens in a background worker on your own computer, so nothing is sent anywhere and the page stays responsive while it runs.

Configure it#

Run lengthdefault 5 minutes

How much simulated time each run covers, in seconds, minutes, hours or days. The unit chosen here applies to the warm-up too.

Discard firstdefault 0

The warm-up: how much of the start of every run to leave out of the figures. It must be shorter than the run length. The gear beside it finds one for you.

Runsdefault 30 — from 2 to 2000

How many times to run the same model. Each run is an independent sample, and the spread between them is what the confidence intervals are built from.

Seedrandom each time — 0 to 4294967295

Where the random numbers start. The runs use the seed and the numbers after it, one per run, so the same seed and model give the same set of results. The dice draws a new one.

Thirty runs is a sensible start. More runs narrow the confidence intervals — roughly, four times the runs halves them — and you can always add more once you have seen the results.

Find a warm-up#

Every run starts empty: no queue, no entity in the system, every server idle. For a while the model is filling up, and the figures from that stretch describe the start-up, not the system. If what you want is the system in steady state, the start of every run has to be left out.

The gear beside Discard first opens Find a warm-up. It runs the model eight times for the configured length and plots the mean cycle time over the course of a run, averaged across the runs — Welch's method. Cycle time climbs while the model fills and levels off once it has; the warm-up is where it stops climbing.

The Smoothing slider evens out the curve: too little and the trend stays buried in noise, too much and everything looks flat. Set Discard first where the curve levels off — a dashed line shows it on the chart — round it up rather than down, and press Use this warm-up to take it back to the batch.

While it runs#

A progress dialog counts the runs finished and how much simulated time that has covered. Two ways out:

• Stop ends the batch early and shows the results of the runs already finished. The run in progress is discarded. It is available once at least one run is done.
• Cancel abandons the batch and returns to editing.

Read the results#

The results open in a panel beside the model, so you can read them against the diagram they came from. Each figure is the mean over the runs, ± a 95% confidence interval: repeat the batch with other seeds a hundred times, and about 95 of the intervals would contain the true value. An interval that is wide compared with its mean is asking for more runs.

• Entities per run — how many entities were completed and dropped in a run, and the share lost.
• Cycle time — the mean, median, 95th and 99th percentile of how long an entity took from its generator to an exit.
• Cycle time distribution — a histogram of the cycle time of every completed entity, pooled across all the runs.

Run 10 more adds ten runs to the set, carrying on from the seeds already used, and updates every figure. It is the quick way to tighten an interval without starting over. Close returns to editing.

Results for each component#

After a batch, each component shows a Show results button. Click it, or the component itself, for that component's own figures per run, each with its confidence interval:

• Generator — completed and dropped entities per second.
• Queue — mean length.
• Server — the share of time busy, waiting and blocked, and completed and dropped entities per second.
• Delay — mean in flight, and completed and dropped entities per second.

The Across runs menu picks one of those figures and shows a histogram of its value in each run — the spread behind the interval, and a quick way to spot a run that behaved unlike the others.

Test the cycle time distribution#

The flask beside the cycle time distribution opens Goodness of fit. Choose a distribution you expect cycle time to follow — exponential with a given rate, or normal with a given mean and standard deviation — and the dialog plots it against the empirical distribution of the pooled cycle times.

It then runs a Kolmogorov–Smirnov test. D is the largest gap between the two curves, marked on the chart where it opens; the p-value is how often a gap at least that large would turn up by chance if cycle times really followed the chosen distribution. A small p-value says they do not.

It is a good check of intuition. An M/M/1 queue, for instance, has an exponential cycle time with rate μ − λ — the M/M/1 library model should follow an exponential with a rate of 0.2 closely once its warm-up is discarded. With many entities the test becomes very sensitive, so even a small departure can give a tiny p-value; look at the chart as well as the number.

Repeat a batch#

The results panel shows the seed the batch started from, with a button to copy it. Type that seed into the configuration of a later batch, with the same model, run length and warm-up, and you get the same results — which is what makes two versions of a model comparable on equal terms: change one setting, keep the seed, and any difference in the results comes from the change.