Earlier this week, together with our partners at fetch.ai we released a driver walkthrough video that lets you come along for the ride during the M-Zone Field trials.  For the first time, Datarella and Fetch.ai have field-tested an AI-powered Deep Parking solution installed at the Connex building complex in Munich. M-Zone provides automated incentives for efficient smart parking in metropolitan areas.  It cuts C02 emissions by providing drivers with real-time options for parking and nudging them with tokenized incentives to park when and where demand is lowest without wasting time or energy driving in circles looking for a spot. Lots of people have asked for more details on how the system works so we decided to draft a technical deep-dive post explain how we built the system and what’s next for M-Zone.

First, let’s dive into a description of the hardware we used to make the M-Zone Smart Parking field trials possible. After that, we’ll delve into the software components and architecture as well as taking a look into how the fetch.ai agents interact with one another and what those interactions mean for cities, parking infrastructure providers, drivers, and the environment.

Edge Nodes

Two edge nodes powered up and scanning for plates. The display between them displays images captured during testing.

For the M-Zone field trials, we deployed two edge computers running computer vision software and fetch.ai autonomous economic agents to the Connex buildings at Frankfurter Ring 81 and 15. These edge computers have two main jobs. The first job is to monitor incoming and outgoing traffic and to read the license plates on incoming vehicles. The second job of the edge nodes is to keep track of the fill state of the parking lot and to publish data about available parking to their associated coordinator agent which is in turn registered on fetch.ai’s Simple Open Economic Framework.

Hardware

• Compute: Raspberry Pi 4 Model B
• Power: Uninterruptible Power Supply (Pi hat) and 1000 mAH Battery Pack
• Connectivity: 4G router with OpenWrt
• Cooling: Heat Sink with GPIO Risers and fans
• Optional Video Output: 7” Display attached to casing with magnets
• Enclosures: Waterproof aluminum boxes that have been modified for cable and camera routing as well as the addition of a plexiglass window for better LTE connectivity
• Various USB A, C, and Micro HDMI cables for routing power and video

The hardware used in the field trials is based on cheap and ubiquitous raspberry pi computers and is intended to provide a plug and play upgrade to “dumb” parking infrastructure.  Deployment is as simple as mounting the waterproof enclosures in a position where they have a good view of the entrances and exits of the parking garage allowing them to compute the fill level of the lot by observing the comings and goings of autos on a constant basis. Our computer vision solution uses the OpenALPR libraries to accomplish plate detection, edge recognition, binarization, deskewing, character segmentation, and finally optical character recognition to read out the license plates.  This enables the nodes to authenticate autos on-the-fly. Future versions will contain a few hardware upgrade options include ruggedized custom enclosures and an improved embedded connectivity solution. The current version performed admirably and passed the field trial with flying colors.

Software Architecture

The real secret sauce doesn’t really come from the hardware though but rather through the software. One of the most critical design choices we made with M-Zone is to host all the cloud-based portions of the system using a Kubernetes cluster for orchestration. This design choice allows the Postgres database and our swagger API to be deployed in a distributed fashion, running on multiple pods within the cluster. This has multiple long term advantages.

It provides options for redundancy at the data state and application layers across multiple nodes located in multiple geographies and using multiple centralized and decentralized cloud/storage options simultaneously. Currently, our K8 cluster is hosted on an AWS EC2 instance but it could be hosted simultaneously across a number of infrastructures in the future. Another key benefit of this approach is the built-in ability to do auto-scaling the database to match load and available resources within the cluster. Building out the system for the M-Zone field trials would have been a lot easier if we had used a less elaborate traditional architecture without orchestration but we believe the investment will really pay off especially regarding the deployment of the IoT nodes. In our field trial, we only had to manage two parking agent nodes but we plan to scale the system and open source it so that such systems can be scaled to cover entire cities. At that scale, it becomes really critical to have an industrial orchestration system that allows you to deploy devices as fast as you can flash SD cards and then never touch them again. Our use of Kubernetes means that we can push updates to the nodes anytime we need to via an “over the air” 4G connection eliminating the need to interact with nodes physically once they’re deployed.

Your micro incentives and real-time parking lot states visualized.

Software Components (from Architecture Diagram above)

• Parking Agents: The parking agent is responsible for the edge processing. Each one runs as a fetch.ai autonomous economic agent inside a docker container running on a Kubernetes pod which is registered as part of the same cluster.  The Parking Agents are responsible for identifying the autos that enter and exit the lot by their license plate and matching those plates against the accounts of registered drivers in a privacy-preserving manner. All the image data remains at the edge to eliminate any possibility for centralized malfeasance and prevents data siloization by design. You can check out our privacy design concept for M-Zone here.
• Coordinator Agent: The coordinator registers as a service on fetch.ai’s search and discovery mechanism for autonomous economic agents (SOEF). This allows for the parking agents to find the coordinator. The agent also has responsibility for dynamically calculating the incentive payments due to the registered vehicles and executing these payments on the fetch.ai V2 Testnet.  It also has the responsibility of periodically converting the issued V2 Testnet micro incentives into FET and sending settlement transactions out to wallet holders.
• Settlement Wallet: We built a custom version of our XSC Smart Wallet to handle the receipt of FET settlement transactions.
• Fetch V2 Testnet CLI Wallet & Account Visualization Web App: In order to handle the micro incentives on the Fetch V2 testnet we utilized a CLI wallet and visualized inputs from both our API and from the current wallet state to provide drivers with a real-time view of which parking lots have the most space and provide the best incentives.
• Dashboard: Just for fun we also built a dashboard that provides an overview of the overall system. This is connected directly to the Postgres Database hosted within the cluster.
• API: A swagger API provides the information consumed by the Account Visualization Web App.

High-Level Sequence Diagram

In the above sequence diagram, you can see that the Parking Agent edge nodes continually look for new images provided by our Kubernetes cluster. Next, they search for available agents on fetch.ai’s Open Economic Forum (OEF), the search and discovery mechanism for autonomous economic agents. The OEF returns the Coordinator Agent address to the Parking Agents after which the Parking Agents are able to register themselves with the coordinator.  At that point, these agents start scanning for license plates.  When they recognize the entry or exit of vehicles, they send a parking event to the Coordinator Agent which acts on the information by updating parking availability broadcast to the wallet via API and also dynamically adjusting the reward ratios and sending the rewards as needed (micro incentives & settlement transactions).

Parking Agent Skills

The parking agents are responsible for all the edge processing. License plate recognition is handled by the third-party library Open Alpr Upon agent instantiation. After the agent starts it performs an OEF search to locate the coordinator node. Once the agent has successfully located the coordinator agent, the agent sends an event packet to the coordinator agent. There are 2 types of events that parking agents can currently trigger.

• Parking events: This event type provides information to link the entrance or exit of autos in the parking agent field of view to timestamps and map those autos to registered user addresses on the fetch blockchain.
• Agent Update: This event type contains a status update from the parking agent. Within its main act function, the agent checks the most recent image saved from the raspberry camera. Any license plates are temporarily stored within the agent memory and deleted following processing. The size of the license plate detected relevant to the frame is also temporarily stored on the edge. Based on whether this frame size is increasing between snapshots or decreasing we can calculate whether a car is entering or exiting the garage. On each iteration, this memory bank is compared with the current image and if the agent detects a new plate, it sends an event update to the coordinator.

Coordinator Agent Skills

The coordinator registers as a service on the OEF which allows for the parking agents to find the coordinator. The agent also has responsibility for calculating the incentive payments due to the registered vehicles. Payments are incrementally made using the Fetchai v2 ledger due to low transaction costs and speed of settlement. Settlement payments are then periodically aggregated at a pre-determined interval to be batched and sent.  Through this process, drivers receive payments of FET that are directly driven by their recent driving and parking behavior.

What’s the Economic Theory at Work?

Neoclassical economic models make a great of assumptions that often don’t hold up in the real world.  Particularly under conditions of information asymmetry and in areas where public goods and externalities are present, “perfect competition” usually breaks down and inefficient markets are the outcome. This is what we currently observe in the parking market and it’s a big part of the reason why parking in cities is so annoying.

Public goods are defined as goods that are both non-excludable and non-rivalrous. Externalities are costs or benefits that are imposed on a third party who did not agree to incur that cost or benefit as part of an economic transaction. Market-based economies struggle to contain negative externalities like pollution and struggle to allocate public goods such as physical infrastructure because the assumptions of “perfect competition” don’t hold in the real world and markets don’t lead to efficient outcomes in the presence of these real-world issues.

At the risk of glossing over too much economic detail, essentially, in order for anything close to an efficient market for parking to exist, we need much better information. The M-Zone parking liquidity protocol is at its core a machine for improving market information levels and providing market participants on both the demand and supply sides of the parking equation with appropriate nudges to incentivize market participants toward more efficient market outcomes.  The result is less CO2 emissions, better utilization of existing parking infrastructure, more efficient permitting processes and less time spent driving in circles looking for parking.

What’s Next for M-Zone Technically?

We’re currently in the process of defining the roadmap for building out M-Zone.  The exact steps aren’t yet locked-in but there are some major topical areas that we can say are on the agenda.

• Self Sovereign Identity-based Authentication
• Payment gateways
• Reservation pathways
• Multichain search and discovery
• Hardware “in the car”
• More strategies for mobile agents and wallets
• Improved UI and driver registration processes

Stay tuned over the next few months.  There’s much more to come!