We recently reported on the unique code Tesla added to its vehicles to identify the region’s crash testing body (and you can see the latest on that here).
In investigating this additional code, we uncovered evidence that Tesla was updating the firmware of vehicles destined for crash testing after they left the factory and before they were crash tested by the crash testing body.
Twitter user green, who is known for exclusively revealing upcoming Tesla features based on code deployed to vehicles in over-the-air updates, discovered this anomaly after obtaining a set of computers from crash tested vehicles that featured unique development code.
Green spoke exclusively with CarExpert and told us that the development code isn’t something you’d find on any vehicle owned by a member of the public.
“In the units I got, some appear to have firmware that is rare or even not present in the public search. While it’s not always possible to say this is development firmware, the ones that do not appear in various firmware aggregators like Teslascope and Teslafi are at least very rare to the point that none of the cars participating in those services and no people online ever mentioned them or a full on ‘special/one-off builds’,” green said.
How does green know Tesla uploaded development firmware to the vehicles destined for crash testing and safety assist testing? They actually stumbled across a set of Model X and Model 3 computers that were removed from destroyed crash test vehicles and then sold through the open market in Europe.
When they were taking the firmware apart, they discovered recordings from the computers that showed the crash tests taking place. This footage matches the public footage released by Euro NCAP of the crash tests. These computers with this footage then obtained development firmware that was never deployed to public Tesla vehicles.
This development firmware was found on one of the Tesla Model X vehicles crash tested in 2019 and on a Model 3 crash tested in 2019. There were even some computers obtained from the same source that showed evidence of modified firmware.
“Some other computers have not been in any accidents as far as I can see, but they came from the same source that provided the ones with crash tests recorded [on video] and they shared various common properties with the definitely crash-tested computers like special provisioning steps to ensure Tesla personnel can access them remotely in a much easier fashion than a regular production car,” green said.
Crash tested Tesla Model X computer showing developer firmware
Green confirmed that while modifying a vehicle’s firmware won’t affect a physical crash test, it has the potential to modify how the vehicle performs during a safety assist test.
“Additionally I’d like to point out that the firmware version on a crash tested vehicle does not matter as long as the crash tested vehicle was not used for other testing prior to the crash test. This is because during crash testing the cars are not running under their own power and all the impact side sensors are covered as part of the protocol so firmware should not be able to affect outcome of those tests. Where firmware can affect outcome is various ADAS test scenarios like emergency braking for cars and pedestrians and the like,” said green.
We wanted to get a better understanding of why this was happening and how modified vehicles were representative of a vehicle a customer could buy. And, the answers surprised us. Following our questions, Euro NCAP started an investigation and came through with the answers we were after.
“To date, these [investigations] have focussed mainly on correlating Tesla’ software updates with the known histories of the test vehicles, publication of ratings, and agreements made with Tesla before testing was carried out,” said Euro NCAP programme director Aled Williams.
“Euro NCAP allows manufacturers to update their vehicles up to the point they are tested. It is to the benefit of car-buyers that manufacturers should seek to offer them the most highly-developed, best-performing systems available. However, Euro NCAP requires that the cars tested should be the same as those in production at the time the safety rating is published.”
According to Euro NCAP, a vehicle rating is only applied to vehicles that meet any of the software or hardware revisions made by a manufacturer prior to testing and often a VIN is quoted on the Euro NCAP website to indicate how and when the rating applied.
In the case of Tesla, and vehicles with updatable software, any software updates that concern safety tests that can be retroactively applied to a vehicle allow it to be granted that safety rating.
We asked Euro NCAP to specifically confirm whether the updates Tesla made to its vehicles to achieve high test results actually made it to customer cars as part of a rating being deployed.
According to Euro NCAP programme director Aled Williams, this is how the sequence of events occurred.
“So far, Euro NCAP has been able to match the chronology of software updates to the selection and testing of vehicles, and the publication of ratings, for Model 3, Model X and Model Y. In broad terms, the ‘cadence’ for all of the cars has been the same and is as follows…and Euro NCAP can confirm that the software updates made by Tesla have aligned with the timeline set out:
- Cars are in production and are provided to customers
- Cars are produced from which Euro NCAP will make a random selection of test vehicles
- The software of these cars is updated at the end of the production line, like that of all other vehicles. This is standard production practice to ensure that cars leaving the line have the latest software installed.
- Euro NCAP selects cars for testing. This is done randomly from the selection of cars offered by Tesla. At the time of selection, Tesla do not know which car will be used for which test.
- At the same time as the above Tesla continue to run their own tests to refine and optimise vehicle performance in Euro NCAP’s upcoming ‘official’ tests. Euro NCAP is not a party to Tesla’s own development tests.
- At a point close to the start of official Euro NCAP testing, the software is ‘frozen’. Tesla starts to roll-out the new software by OTA to customers’ vehicles, a process which can take some weeks to complete (owing to the number of cars which have to be updated).
- At around the same time, Euro NCAP’s test vehicles are also updated with the same software.
- Official tests are performed and a rating is generated, based on the results of those tests.
- Testing can take several weeks, and the post-test inspection and follow-up work also takes time. From the start of testing to publication of the result can take some three months.
- Before the publication of the result, Tesla roll out the new software in production cars. Those cars produced before the update is in production are updated by OTA.
- Euro NCAP publishes its rating, having tested cars with the same software that is used in production vehicles and which has been retrospectively deployed to previously-built vehicles.”
So it appears that Tesla is actually using the crash body tests to further improve the vehicles that customers were buying and deploying these updates to customers free of charge as over-the-air updates.
While Euro NCAP admits it was unable to verify what the development code within the test vehicles contained, Tesla assures them the elements of the development code that related to safety were deployed to customers as part of over-the-air updates.
According to Williams, Euro NCAP hopes to publish tested software versions in the future to better help align features and software for customers.
Our earlier story related to Tesla being found by Twitter user green adding extra code to vehicles that indicated the local test body – i.e. ‘euroncap’, ‘ancap’, etc. – you can find out more about why this code was added and what effect it has on vehicles in our original story.
