How satellites and artificial intelligence monitor vegetation near tracks

By Frédéric de Kemmeter – Railway signalling and freelance copywriter – Suscribe my blog
24/08/2020 –
(Version en français)
🟧 Back to homepage 🟧 See our brief news

The technology to understand vegetation may seem incongruous. Yet, applications from the fields of Artificial Intelligence and Data can help to better evaluate the vegetation on a given area, to understand the complex dynamic systems of this vegetation and to take corrective action when necessary.

Our environment is a complex structure. Often the processes acting within it are individually understood, but their interactions and feedbacks among each other are difficult to grasp with classical approaches. The human influence on the environment also increases the complexity. An example of the complexity of our environment is the operation of a water-cooled power plant. This power plant takes cooling water from a river to keep the plant at operating temperature and returns the heated water to the river. If water becomes scarce during a drought or if the available water is too warm, the power plant has to be shut down. Therefore, the control system of the plant must be able to predict the development of the thermal status of the water and at the same time be able to recognize its own influence on the status of the environnement. This task can only be solved by using a specialized artificial intelligence (AI). It must be able to recognise climatic boundary conditions at an early stage and independently develop action scenarios to prevent the plants from being shut down. The railways must have the same reasoning in order to offer a perfectly functioning network.

According the british infrastructure manager Network Rail, resilience of a network is the ability of assets, networks and systems to anticipate, absorb, adapt to and / or rapidly recover from a disruptive event. Following multiple failures in the winter of 2013/14, the Department for Transport concluded that the physical resilience of the network is progressively strengthened in future years. These failures were caused by major problems of terrain stability, caused by water infiltration or less than optimal vegetation.

(photo Network Rail)

October 2016, Network Rail launched an industry challenge. The challenge was to demonstrate solutions, using satellite data, which could have alerted Network Rail to conclusive asset deterioration and service ability loss in advance of failure at the locations of the challenge examples. For Network Rail, the challenge clearly demonstrated the needs could not be met and a thin linear infrastructure like a railway line, consisting of slopes predominantly covered in vegetation cannot yet be adequately monitored from spaceExperts said at that time that « although techniques are considered to have already reached the operational level, it is apparent that in both research and practice we are at present only beginning to benefit from the high resolution imagery that is currently acquired by the new generation radar satellites. It is very difficult to measure displacements exceeding few tens of cm/year and strong nonlinear deformations. » But the technology has since greatly improved.

Built in the 19th century, the railway network is totally inserted in a vegetation that has regained its rights over the decades. In Germany, around seventy percent of DB’s tracks run through areas with tree populations. In recent years, DB, like other railroads, has significantly expanded the vegetation care along its lines to make them more storm-proof. With around 33,400 kilometres of railway lines, this means millions of trees and plants that need to be examined for their storm resistance due to their location, condition or shape. In Germany, some storms have already brought a third of the rail network to a standstill, particularly in the north and east of the country. Between 2015 and 2017 alone there were over 830 train collisions with vegetation caused by storms, costing millions. The phenomenon is therefore being taken very seriously. The phenomenon is therefore being taken very seriously.

After « Xavier » storm… (photo V.Boldychev via wikipedia)

Infrastructure network operators mainly use manual, inflexible, tightly focused solutions, e.g. the use of cars or helicopters to monitor infrastructure networks, despite the availability of satellite data. These manual monitoring processes are often inefficient and lead to high network operating costs. Large-scale monitoring or inventory of networks covering an entire country can take years with the methods used so far.

That is why Deutsche Bahn has relied on professional vegetation management. Around 1,000 DB Netz employees are involved in all aspects of forest and vegetation management. From now on, digital technology will help them in their task. So Deutsche Bahn is now recording trees from outer space to ensure a storm-proof vegetation stock. So Deutsche Bahn is now recording trees from outer space to ensure a storm-proof vegetation stock. Satellites record the number of trees, the distance between vegetation and tracks, and the height at which trees grow along the railway network. For this programme, Deutsche Bahn has significantly increased financial resources, and €125 million are available annually for vegetation maintenance. Of course, DB Netz does not do this alone.

The German network operator relies on LiveEO, which participated in 2018 at the Mindbox, a start-up incubator of Deutsche Bahn AG which gives innovators the opportunity to test and fast-track their ideas and products inside Deutsche Bahn. With this program Deutsche Bahn supports young entrepreneurs and innovators. DB supports for 100 days start-ups that have ideas for improvements for railways – among other things with extensive coaching as well as 25,000 Euros start-up capital. This is complemented by live testing of prototypes and coworking in DB mindbox.

As explains Sabina Jeschke, Director of Digitalization and Technology Deutsche Bahn, the idea is to anticipate a possible incident as much as possible. Where do the railway’s vegetation experts urgently need to go? Which trees and plants are unable to cope with increasingly extreme weather conditions? Which vegetation is more resistant than the other? What should be maintained or planted as vegetation to consolidate a slope or trench? To what level can vegetation grow before causing skidding problems?

(photo DBAG)

DB has been regularly sending drones up his network since 2015. A fleet of 20 multicopters is now in service throughout Germany. They are equipped with cameras – mostly for videos, high-resolution digital images or infrared recordings. For vegetation control along the railway lines, drones fly over the areas with infrared cameras. The images are checked for diseased trees that could fall onto the track in strong winds. The photographic analysis generates between 3 and 5 gigabytes of data per kilometre of track. The data is evaluated using Big Data Analytics. The procedure is very reliable: Trees that are not storm-proof are detected with a probability of 95 percent.

LiveEO does not have its own satellites and must then use optical images of the planet taken by Airbus’ Pléiades satellite. The Pléiades satellite constellation is comprised of two very high-resolution optical Earth-imaging satellites that provide coverage of the planet with a repeat cycle of 26 days. This high-resolution satellite data was in the form of stereoscopic images, allowing LiveEO to compare and derive height profiles of trees from it. LiveEO uses multi-band imagery at various resolutions to distinguish vegetation and network assets, stereo imagery to estimate tree heights, difference detection to identify changes, and interferometry to detect subsidence and other ground distortions at the cm level. LiveEO is developing its own inhouse training data for machine learning algorithms.

Being able to accurately detect tree height near railway lines improves safety, identifies high and low-risk trees and reduces the cost of tree clearing. Deutsche Bahn can now save around 25 percent of their operational expenses by improving efficiency.

LiveEO is now producing a digital map of the forests along railway lines for Deutsche Bahn. Satellite images are used to precisely record tree populations, the distance between vegetation and tracks, and the height at which trees grow. This enables DB to identify trees that are particularly susceptible to storms even better and to treat them in good time. The peculiarity of LiveEO is that we now have only data about vegetation, whereas the drones capture « everything near the track », without distinction, which required a lot of work to clean the maps, which is very time consuming. This demonstrates the importance of incubators, which make it possible to capture ideas that cannot always be generated within a large, highly hierarchical company such as a railway company. Especially in digital subjects, which require knowledge that has little to do with the railway business.

The technology of LiveEO helps to prioritize and optimize not only vegetation management, but also to review processes in the event of service disruption. The amount of data is such that only an AI can synthesize it in record time and reduce the duration of disruptions. The combination of enterprise resource planning, satellite imagery, machine learning (ML), and artificial intelligence (AI) provide better answers to vegetation management challenges.

  • High-definition satellite imagery combined with multiple sensors can eliminate the laborious process of personnel on the field manually scouting through the terrain and collecting data.
  • ML/AI algorithms can process data to identify specific spots where vegetation might encroach or interrupt operations and help railway teams pinpoint specific spots for trimming or removing vegetation.
  • Dashboards can spot alerts raised by the algorithms.
  • Simultaneously using tree species’ inventory and weather data can further optimize systems.
  • Integrated ERP solutions can enable further automation for engaging teams by planning and scheduling work orders and tracking execution updates within parameters set by the budget.
  • ERP solutions can be combined with mobile field solutions to enable field teams to capture details during vegetation trimming.

LiveEO actually sells two products. The first is the « frontend » which is designed for decision makers and management and provides an overview of the entire network. The second is a mobile app allows on-site staff to take on specific tasks from management and document progress in processing. « This enables transparent communication between all user levels, » says Sven Przywarra, the co-founders of LiveEO GmbH.

But DB also focuses on managing major disruptions, and in particular on providing passengers with information. If there is a disruption that prevents trains from continuing or diverts them, passengers’ need for information rises abruptly. For Deutsche Bahn, the main concern then is not to create an information vacuum and to keep customers informed as quickly and continuously as possible. It is then necessary to have the right information as soon as possible. Where exactly is the « tree in the track »? Which lines are blocked? Which trains are stopped? Where are trains parked? What does this mean for rail operations as a whole? This is where LiveEO comes into play again. « The satellite data from the startup could be of enormous help to us even in the event of a fault. This year (2020), we plan to test whether the technology can give us a quicker overview of the situation on the lines in the event of a disruption than is currently possible. The aim is to keep the phase of uncertainty for customers as short as possible and to issue a forecast as soon as possible, » explains Sabina Jeschke. Which also shows the possibilities of AI within the railways: « Language assistants or chat bots could also be useful, because the intelligent systems help employees to quickly cope with the sudden increase in the need for information. » 

(photo pqsel)

LiveEO is a good example of a company who apply analytics to the huge amount of earth observation data to provide a valuable service that is safer and less expensive than the traditional approach to railway line vegetation monitoring. Nowadays there is a plethora of satellite imagery with resolution as high as a cm and a frequency of several times a day. Satellite imagery is becoming increasingly competitive which is lowering the cost of imagery. There exists other innovative applications that use satellite-based radar interferometry to detect subsidence, which are great dangers for railway network.

With nearly 2000 live satellites circling the globe, getting a detailed picture of any part of the Earth’s surface is no longer a problem. The challenge is working out what to do with all these images. Analysis of geospatial data is where the next big space bucks are to be made. There is no doubt that the next few years will see a sharp increase in the combined use of satellite images and AI to increase the safety of rail network asset management and prevent future hazards, including the real-time integration of weather data.

Sources :

2018 – Network Rail – Earthworks Technical Strategy

2019 – Sifted – Maija Palmer – Airbus launches satellite data platform

2019 – Innoloft – Lucia Walter – LiveEO is the startup of the week 36: Innovative Infrastructure Monitoring from Space

2019 – Okeanos Consulting & Ruhr-Universität Bochum – Dr.-Ing. Benjamin Mewes und Dr.-Ing. Henning Oppel – Künstliche Intelligenz in den Dienst von Mensch und Umwelt stellen

2020 – Geof Zeiss – Using satellite imagery to prioritize vegetation management for utilities

2020 – Deutsche Bahn – Prof. Dr. Sabina Jeschke – Ruhe in den Sturm bringen

2020 – Eisenbahn Journal – Stefan Hennigfeld – Vegetationspflege mit künstlicher Intelligenz

(photo Reinhard Dietrich via wikipedia)

24/08/2020 – By Frédéric de Kemmeter – Railway signalling and freelance copywriter
Suscribe my blog