Skip to content
Connected RailsConnected Rails

Detecting from the aerial imagery

Part of the route editor.

The photograph the editor drapes over the ground is a survey of the real place, and most of what a builder does is transcribe it: every car in the station car park, one click each. File ▸ Detect from imagery… has a local model do it.

Everything runs on this machine. The runtime is tract, a pure-Rust ONNX implementation compiled into the editor — there is no service to sign up to, nothing is uploaded, and a module can be built on a train with no signal.

Where it looks is the point. A model let loose on a whole module would fetch a square of imagery a hundred times bigger than the line and take an hour about it. So the dialog asks for one of two areas:

  • Along the track — a corridor of a stated width, which reaches the station forecourt and the goods yard without walking the fields behind them, or
  • In the drawn area — the ring drawn with the AI area tool, or the circle the select tool last grew round something.

Both carry the same second condition, Keep clear of track: whichever area was chosen, nothing is placed within that many metres of a rail. A car standing in the four-foot is worse than no car at all.

And nothing is placed on a carriageway. A photograph shows the cars that were driving when it was taken as readily as the ones that were parked, and nothing in the picture tells them apart — but a parked car left in a running lane is a road blocked for the simulator’s own traffic, which is worse than a car park a few cars short. So every find whose middle falls inside a road’s width is dropped. Measured to the kerb and no further: a car at the roadside has its centre just beyond it, and kerbside parking is most of the traffic beside a street. A car park this dialog paved on an earlier run carries parking and is not a street — otherwise a second run could not fill the bays the first one found. The rule can only hold to roads that are in the file, so a module whose roads have not been imported yet is told so in the report rather than quietly given a lorry in the fast lane.

The run happens on a thread of its own with a progress bar and a Stop that means it, and nothing is written until Commit — the report says what was found and what is installed to place it with, and the whole run is one undo step.

For a module being rebuilt from its sources by a script there is --detect-run, beside import-module:

Terminal window
trainsim-route-editor mods/example/lines/boerde.ron --detect-run \
[--corridor 80] [--keep-clear 8] [--model dota-obb]

It runs the same detection along the track, commits it and writes the line file back, printing the progress and the report to the log. The dialog stays the way a person should do this — reading the report before a few hundred objects go into a module is the point of it — and the flag is for the case where nobody is sitting in front of it.

Car parks are not detected. They are the cars: a cluster of them is fitted with a rectangle along the rows they stand in and paved with an unmarked asphalt area, which is an ordinary road afterwards and can be dragged about like one.

A crown detector run over the same imagery plants the wood that is actually there. A class of a model can say it is a tree (kind: Tree in ai.ron), and then a find does not become an object bolted to the track graph — it becomes an ordinary row in the module’s tree list, at the place the crown was, in a species from the installed tree mods, and grown to the size the crown was measured at. A twelve-metre crown gets a twelve-metre tree: the species is drawn from those whose own crown is within half again either way (mods/trees ships every species as three individuals — young, grown, old) and the one that is drawn is then scaled the rest of the way, within a band, because a spruce squeezed to a third of itself is not a young spruce but a spruce seen through the wrong end of a telescope.

The crown detector ships with the game: models/deepforest-tree.onnx, which is DeepForest — the crown model of the field, a RetinaNet trained on the NEON airborne survey and MIT licensed. It reads a window at five centimetres a pixel, which is not the resolution it was trained on but the scale at which a crown arrives the number of pixels across it expects; the finer the imagery a provider gives, the better the crowns come out.

What kind of tree comes from the model where the model knows. Every crown detector published today is single-class — DeepForest included, because tree is what aerial training sets are labelled with — so where a class names a conifer tag as well, the crown itself is asked: needles are dark, blue-green and hard-shadowed on one side, because a conifer is a cone; broadleaf foliage is a flatter, yellower dome, and in autumn frankly orange. Both tests have to agree before a crown is called a fir, so the guess leans to the commoner tree. It is a guess and the editor says so — a model with species classes of its own leaves conifer empty and is never second-guessed.

Which trees, in the end, is a question the photograph cannot always answer, and the dialog says so. Species offers “as detected” — the model’s class, and the crown’s own look where the model knows only one — or any stand the installed mods describe (stand-nadelwald, stand-laubwald, stand-mischwald, …, the same stands the forest brush plants from). Naming one overrules the guess for the whole run, and the size goes with it: the stand is then planted the way the forest brush plants it — any of its members, at its own size give or take a third — because a crown detector reading a provider’s imagery reports the sunlit top of a young conifer rather than its spread, and a spruce wood planted at that measurement would be a wood of saplings. Where the species come from the model, the crown decides both the member and the size, which is the point of measuring it.

Everything that follows is what follows for any other tree. An AI-planted wood is drawn by the same vegetation instancer, is picked by the select tool like a tree planted by hand, joins a multi-selection through Ctrl-click or the select circle, and goes with one Delete — single trees or a whole marked stand. A run is one undo step, and nothing about a wood the model found is harder to take back than a wood a person drew. A second run over the same ground changes nothing rather than doubling it: Keep trees apart is the distance at which a crown counts as already planted.

(
id: "deepforest",
name: "DeepForest (Baumkronen)",
file: "models/deepforest-tree.onnx",
input: (width: 768, height: 768, mean: (0.485, 0.456, 0.406), std: (0.229, 0.224, 0.225)),
head: Retina(confidence: 0.3, iou: 0.1),
classes: [
(
name: "Tree",
kind: Tree,
place: "laubbaum", // the tag a broadleaf crown is planted from
conifer: "nadelbaum", // …and the one a dark, shadowed cone gets
span: Some((2.5, 26.0)),// crowns this class covers [m]
),
],
ground_sample: 0.05,
)

Retina is the second output layout the editor reads, and it is there because every crown model worth having is one: a torchvision RetinaNet emits logits per anchor and offsets per anchor, with no boxes at all — an offset is measured from an anchor, and the anchors are not in the file. So the shipped model contains the backbone and the head and nothing else, and the editor rebuilds the anchor grid from the input size. tools/vision/README.md says why that is the better half of the bargain, and Boxes remains what an Ultralytics-exported crown detector of your own would use.

The models are data, not code: ai.ron, written next to imagery.ron the first time the dialog is opened, lists what the editor knows how to talk to — the file, the input size, the head, the ground resolution the model was trained at, and what each of its classes is worth on a module:

(
id: "dota-obb",
name: "YOLOv8 OBB (DOTA v1)",
file: "models/yolov8n-obb.onnx",
input: (width: 1024, height: 1024),
head: Oriented(confidence: 0.25, iou: 0.45),
classes: [
// …the model's own list, in the model's own order.
(name: "large vehicle", place: "lorry", size: (9.0, 2.5)),
(name: "small vehicle", place: "car", size: (4.4, 1.8)),
],
ground_sample: 0.3, // m per pixel the model was trained on
)

place is a tag, and the editor places an object carrying it from whatever mods are installed — mods/cars brings seven European vehicles, from a Kleinwagen to a Kastenwagen, built by tools/cars/ out of generated FBX models: plinth removed, interior removed, windows turned into dark glass where that is the better picture, and four levels of detail each. A detector for level crossings, containers or solar farms is therefore an entry in this file and a mod with objects tagged for it, and no Rust at all. size is the real footprint of the class: a “car” eleven metres long is two cars the model ran together, and it is dropped rather than placed. Where a factor of two around one size is the wrong rule — a crown is anything from a three-metre thorn to a twenty-five-metre oak and both are right — the class states the range outright as span instead.

Which object, of the ones carrying the tag, is decided by how long the find is. One tag holds more than one size of thing — lorry here is a 4.82 m Transporter and a 6.30 m Kastenwagen — and taking either at random stands half the vans a metre and a half out of their bays. So an object states its footprint (Track objects) and nothing longer than the space goes into it: below about six metres the Kastenwagen is simply not a candidate. Among what does fit the choice stays what it always was, decided by the place itself, so the same imagery always draws the same car and a row of bays is still a row of different ones rather than five copies of the largest estate.

Two detectors ship with the game, in models/, so a fresh clone can read its own imagery with nothing fetched and nothing signed up to:

file finds from licence
models/yolov8n-obb.onnx cars, lorries Ultralytics YOLOv8n-OBB on DOTA v1 AGPL-3.0
models/deepforest-tree.onnx tree crowns DeepForest (Weecology), NEON survey MIT

They are Git LFS objects — git lfs pull on an old clone — and they are converted from the published pre-trained models by tools/vision/export_models.py, which is the whole of what was done to them.

The two licences are not the same, and the car one is the loud one. Shipping yolov8n-obb.onnx means redistributing an AGPL-3.0 work; the EUPL this repository is under names AGPL-3.0 among its compatible licences, so the combination is provided for, but the combined work then travels under the AGPL. DOTA itself is released for academic research. If that does not suit how you intend to distribute this, delete the file — the editor then reports the weights as not installed and everything else, the tree detector included, carries on. models/LICENSES.md has the full record and tools/vision/README.md how to put another detector in its place.

Bringing your own is the same two steps it always was — an entry in ai.ron, and the file where the entry says:

Terminal window
pip install ultralytics
yolo export model=yolov8n-obb.pt format=onnx imgsz=1024
mv yolov8n-obb.onnx models/ # next to ai.ron