by Simon Zehentreiter | 19 December 2024 | AI, Blockchain, COSMIC-X, Gaia-X
In the first part of this Cosmic-X blogpost series, we evaluated various blockchain platforms for their suitability in Industry 4.0 and explained why we chose the Secret Network with its confidential computing capabilities. Today, we’ll explore how we use the Secret Network to secure machine data integrity from its origin to its consumption.
Need for Data Integrity
Securing data integrity in Industry 4.0 is crucial because systems and devices rely on accurate data to function effectively. Tampered or incorrect data can lead to poor decisions, operational failures, and vulnerabilities in key sectors like manufacturing and logistics. With IoT and AI driving Industry 4.0, maintaining data accuracy ensures reliable operations, protects sensitive information, and prevents cyber threats that disrupt businesses and critical infrastructure.
Anchoring data close to its source is essential for securing integrity across the entire data processing chain, which often involves multiple distributed systems. For machines, this means securing the data before it leaves the device. At the same time, the system must protect the anchored data from tampering after export. Blockchain’s immutable nature aligns perfectly with this paradigm. That’s why we built a Wallet Service on top of the Secret Network. This service integrates seamlessly into any machine to secure its data integrity in a decentralized and privacy-preserving manner.
Wallet Service
The Wallet Service acts as a gateway for communication with the Secret Network. It deploys onto any machine infrastructure that supports Docker. By using the Wallet Service, machines interact directly with the blockchain and its smart contracts. The service assigns each machine a unique identity through a public-private key pair. With its private key, the machine signs and broadcasts transactions to anchor its data on the Secret Network. The blockchain’s encryption ensures that no unauthorized third party can access the data. For details on how the network reaches consensus despite encryption, refer to our previous post.
Integration
To simplify integration, the Wallet Service offers a straightforward REST API with two endpoints. The ingress endpoint accepts a batch of data in a defined structure for anchoring. After receiving the data, the Wallet Service hashes it and stores the resulting hash in the service’s smart contract through a transaction on the Secret Network. This process creates an immutable fingerprint, allowing anyone to verify the integrity of a data batch through the Wallet Service’s verification endpoint. Since data verification typically occurs in systems other than the one that supplied the data, the Wallet Service supports deployment anywhere. In distributed data processing scenarios like Cosmic-X, entities that consume data instantiate a Wallet Service to verify data integrity before making decisions. For example, an AI service provider might deploy a Wallet Service in its cloud environment to verify data before using it for training or inference.
Requirements
Two conditions must be met for this workflow to function: first, the verifying Wallet Service must have the appropriate viewing key from the machine that supplied the data. Otherwise, it cannot decrypt and query the fingerprints stored in the smart contract. Second, the format and schema of the data batch must remain standardized across the processing chain. To achieve this, we developed a Data Integrity Protocol as the foundation of the Wallet Service.
Data Integrity Protocol
To anchor and verify data batches reliably, the Wallet Service requires a standardized protocol. Both the data anchoring and verification processes must adhere to a common data format, schema, and canonicalization standard. For Cosmic-X, we chose JSON as the data format and RFC 8785 as the canonicalization algorithm. Canonicalization ensures reliable cryptographic operations on JSON data by defining methods for handling whitespace, data types, and objects.
Batch Structure
Considering use case requirements and the limitations of edge and cloud environments in Cosmic-X, we define a data batch as one hour’s worth of sensor data collected from a machine. The figure below shows an extract of a data batch collected from one of the use cases. The batch includes a metadata object used only for the Wallet Service’s business logic. This metadata contains key-value pairs such as the batchId and placeholders for the payload hash and the transaction hash on the Secret Network blockchain. The payload, which the system hashes during anchoring, consists of discrete sensor measurements. Each measurement uses a composite key created by concatenating the variable name with the Unix timestamp of its recording. The measurements include key-value pairs for variable name, timestamp, absolute value, and data type.
The batchId is the most critical part of a data batch. Since the Wallet Service uses it to anchor and later locate the data batch for verification, the batchId must be unique. In this setup, the batchId combines a machine ID with a Unix timestamp representing the time range of measurements in the batch, rounded to the nearest hour. For example, if machine 2080839 collects measurements from 11:01:23 to 11:59:43 on May 16, 2024, the batchId becomes 2080839_1715853600.
In the next post, we’ll showcase how we integrated the Wallet Service with three live machines and an AI service to enable secure and accurate anomaly detection in machine components.
Curious how our blockchain-based data-integrity solution can help your business? Check out our one-pager for a quick overview of its key benefits!
by Simon Zehentreiter | 8 November 2024 | AI, Blockchain, COSMIC-X, Gaia-X
With the Cosmic-X project nearing its conclusion, it is finally time to lift the curtain on the blockchain solution that Datarella has built over the last two years to enable confidential computing and data sharing in Industry 4.0. In this first entry of a series of technical posts about designing, implementing, and integrating an edge-to-cloud blockchain solution, we discuss the evaluation process for selecting a suitable blockchain platform for Cosmic-X and how that platform operates on a protocol level to provide an open, transparent, and secure infrastructure for industrial use cases.
Evaluating Blockchain Platforms
Today, many different blockchain platforms exist, but their suitability for industrial use cases remains specific or, at times, limited. To achieve the best match between the requirements of Cosmic-X and the possibilities of blockchain technologies, the team conducted an extensive evaluation process. This evaluation compared both private and public blockchain platforms based on security, privacy, scalability, and interoperability.
Current-generation blockchain platforms predominantly perform well in security and scalability, yet privacy and interoperability often fall short. To achieve privacy in industrial scenarios like Cosmic-X, organizations have almost exclusively used private or consortium blockchains such as Hyperledger Fabric in the past. However, these approaches inherently involve high infrastructure costs for the operating parties, as well as centralization and limited interoperability. In contrast, public blockchains offer resilience, cost efficiency, and a degree of interoperability. Though only recently have they started focusing on privacy and data protection. Blockchain protocols with confidential computing capabilities remain relatively new and untested. Nevertheless, when weighing the advantages and disadvantages of the two approaches, a privacy-focused public network emerges as the preferred solution in an industrial context.
For a public network to meet Cosmic-X’s privacy and data protection requirements, it must support the multi-tenancy paradigm. Multi-tenancy enables a single instance of a software application to serve multiple clients while ensuring logical isolation. Different clients share an underlying infrastructure, which optimizes resource use and reduces infrastructure costs. Further, it enhances efficiency in data access, management, and collaborative data sharing.
Through this evaluation, the Cosmos-based Secret Network emerged as the blockchain platform best suited for Cosmic-X. The Secret Network functions as a public blockchain specifically developed for confidential computing. By combining established encryption techniques with trusted execution environments, it provides so-called Secret Contracts. This type of smart contract establishes consensus on computation without disclosing incoming or outgoing data. Integrated access control mechanisms enable third-party access and create an auditable processing chain. Thus, the Secret Network satisfies the need for multi-tenancy capability while retaining all the benefits of a public network.
How the Secret Network Works
The Secret Network leverages Intel Software Guard Extensions (Intel SGX) to create Trusted Execution Environments (TEE) that enable Secret Contracts. These smart contracts, based on the CosmWasm framework, allow for fully private computation of data. Outside a TEE, the transaction payloads and the network’s current state are encrypted at all times. Only the data owner and an authorized third party can decrypt and view data inputs and outputs. A combination of symmetric and asymmetric encryption schemes—ECDH (x25519), HKDF-SHA256, and AES-128-SIV—achieves this end-to-end encryption. Each validator in the network must run an Intel SGX-compatible CPU and instantiate a TEE that follows the network’s rules.
When an encrypted transaction arrives in the shared mempool of the network, a validator forwards it to their TEE, where a shared secret is derived and used to decrypt the transaction. The WASMI runtime then processes the plaintext input. Finally, the validator re-encrypts the updated contract state and broadcasts it to the network through a block proposal. If over two-thirds of the current network voting power agree on the result, the network appends the proposed block to the Secret Network blockchain.
For access control, the Secret Network offers Viewing Keys and Permits. A Viewing Key acts as an encrypted password that grants a third party permanent access to data related to a specific smart contract and private key. A Permit allows a more granular approach, restricting viewing access to specific parts of data for a set period. Consequently, despite its encrypted nature, the network remains fully auditable.
In the next post, we’ll explore how we leverage the Secret Network to secure machine data integrity directly from its point of origin to its consumption by a Machine Learning Model.
by Kira Nezu | 29 March 2019 | Blockchain
Cryptography vs Cryptocurrency – one enabling the other
For many, the word ‘crypto’ brings Lamborghini’s, neckties with Bitcoin signs printed on them or really bad hip-hop, but up until some years ago, the original meaning was not crypto-currency, but cryptography. That is also the topic of this blogpost. Consider it a gentle introduction into a very specific branch of cryptography – zero-knowledge proofs (or ZKP) – and why blockchain has helped bring them back into vogue.
What are ZKP – intuition
So, let’s start off with the basics, what are zero-knowledge proofs? I’ll explain it through three common analogies with varying complexity, they all describe the same concept but appeal to different audiences.
1. This analogy is from StackExchange. Imagine your friend, Alice, tells you that she has a super-power. An amazingly useless super-power, but still. She can count all the leaves on a tree in your garden in front of your house in a few seconds! Of course, you don’t believe her, so you ask Alice to prove it. We’ve now created two roles that are omnipresent in ZKP, a prover (your friend Alice) and a verifier (you in this case). She proposes that she closes her eyes, you can then choose to either remove a leaf from the tree or not, and finally she can open her eyes. Now, to prove her super-power, she has to tell you whether or not you removed a leaf from the tree. If she’s wrong she failed to prove anything, but if Alice is right, you realise that she had a 1/2 chance to guess correctly and was just lucky. So you repeat the experiment, now if she’s right again, she would have had to have been right two times in a row, meaning her odds of being guessing correctly were 1/4. (At least assuming independence of events). This goes on and on until you are sufficiently convinced of her super-power being real. In this scenario, you didn’t learn HOW she does her magic counting of leaves, but you’re very sure that she know how to do it. There was ZERO KNOWLEDGE transferred from Alice to you regarding the procedure itself. Additionally, there was no, or a very small possibility for you, being honest, of not believing in Alice’s capability, and she couldn’t have convinced you without actually having the super-power. These three criteria are called ‘zero-knowledge’, ‘completeness’ and ‘soundness’, respectively, and are a part of all ZKP.
2. This one is from the booklet “Applied Kid Cryptography or How To Convince Your Children You Are Not Cheating” by Naor, Naor and Reingold. It relies on the game called “Where’s Waldo?” or “Where’s Wally?” in the UK-version. The goal of the game is to find the image of Waldo on a page filled with other things and figures. Let’s assume Alice and yourself are playing this game together. All of a sudden, Alice exclaims “I found Wally!”. Aggravated with jealousy you scream out “So prove it!”, (first the revelation of the leaf-counting super-power and now this!?). So how can Alice prove her knowledge of where Waldo is, without revealing to anyone else where he is? Simple, she takes a big cardboard with only a cut-out in the middle, just the size of a Waldo-image. As you close your eyes, Alice places the cardboard over the open pages of the Where’s Waldo?-book exactly so that only Waldo can be seen through the cut-out. You can verify that Alice knew where Waldo is, without learning where on the page he is. Again, this satisfies our three properties of zero-knowledge, completeness (you have to believe Alice found Waldo given the information she presented to you) and soundness (Alice couldn’t cheat by randomly placing the cardboard on the book except by being extremely lucky).
3. Now, my favourite example from a highly recommended blogpost by Jeremy Kun is more in the theoretical space. Instead of a difficult problem like counting leaves or finding Waldo, we now have the provably difficult and more formally defined problem of proving that two graphs are isomorphic. Let’s unpack that:
– A graph, G, is defined by a number of edges connecting the vertices of the graph. Thus a graph G = (V,E)
– Each edge can be represented as the tuple (u,v), where u and v are integers between 1 and the number of vertices of G, n.
– Given two graphs G = (V,E) and G’ = (V’,E’), they are isomorphic if there exists a couple of functions f: V->V’ and g: E->E’ such that f associates each value in V with exactly one element in V’ and vice versa. Correspondingly, g associates each value in E with exactly one value in E’ and vice versa.
source: Jeremy Kun’s blog on Math and Programming – https://jeremykun.files.wordpress.com/2015/11/gi-example.png?w=587&zoom=2
Intuitively, this means that graphs are isomorphic if we can transform one into the other by simply moving around the vertices, not adding or removing any edges and ending up with two identical figures. This is not exactly rigorous, but still somewhat accurate for our purposes.
Now, for the zero-knowledge part! Given two graphs, there’s no easy or efficient way of finding out if they are isomorphic. (If you find a way, let me know.) So, let’s say Alice knows that there exists an isomorphism between them, but she doesn’t want to reveal her isomorphism to you. She does this by taking e.g. G and mixing V. Then she sends you her newly formed isomorphic graph, called H. Alice additionally saves the permutation she did on G for later.
After having received H, you flip a coin with equal probabilities and depending on the outcome you give Alice a challenge. Heads, and Alice should provide you with the inverse, or backwards, permutation which gave her H. It should then give you G. If tails, Alice should provide you with her secret isomorphism, f composed with the permutation. This should now give you G’ when applied to H.
Given either of those permutations, you should now be able to verify that Alice possesses a ‘secret’ isomorphism. Additionally, you haven’t learned anything about the solution since you only received a uniformly random permutation or two uniformly random permutations composed which gives another uniformly random permutation.
Why ZKPs are interesting to blockchain
Ok, now that we’ve understood a bit what ZKP means, let’s see why it is interesting for blockchain technology. The most obvious area of application is of course privacy. Being able to prove something without having to reveal any information about the subject sounds like utopia for almost everyone with an eye on the current state of affairs in big data applications of corporates and states. A second, less obvious type of application is for scaling in blockchains. This relies on the fact that a proof of knowledge can be more succint, from a storage point of view, than the information it’s proving. Let’s look at some use cases of both application areas in more detail:
One of the first live applications of ZKP in blockchain for privacy was by ZCash – a cryptocurrency where the ‘knowledge’ being proven is that the sum of outgoing transactions are equal to the sum of incoming transactions (ZCash uses a UTXO model), that the sender has the authority to spend the coins being sent and finally that the private keys of the incoming ‘notes’ are effectively locking the whole transaction from being modified without the keys in question.
Another use case of ZKP for privacy is by Sovrin, who mainly uses regular public key cryptography and a fairly clever protocol to issue verifiable credentials such as “possession of a valid driver’s license in EU”. Then they apply a type of ZKP called accumulators to prove non-revocation of that very credential in a very succint manner. This was initially researched by IBM in the so-called idemix, back in 2007, but lacked an adequate platform to store the non-revocation lists in a persistent, trustless manner. Until blockchain arrived.
Generally speaking, ZKP can be used for a wide range of privacy-preserving applications, especially when it comes to the topic of identity, things such as range-proofs whereby it can be proven that one’s age is within a certain range (e.g. 18-65) without revealing the actual age. Or it can be proven that one is a resident of the EU without revealing in which country exactly.
One of the most pressing issues of public blockchains these days (and admittedly since some time) is that of scalability. Interestingly, ZKP may have a solution for this. Like ZCash, another privacy-focused cryptocurrency Monero implemented ZKP. However, Monero was using a different algorithm called RingCT to hide transaction information. It didn’t rely on the often criticised ‘trusted setup’ of ZCash (more here) but therefore had a very large transaction size resulting in low throughput. This was improved greatly by the application of so-called bulletproofs (also a type of range-proofs actually) in October 2018. This meant that the average transaction size was reduced by at least 80%, and the fees accordingly.
Even more extreme measures are being built by the coda team who aim to recursively compress an entire blockchain into a 20kB ZKP. Their CTO Izaak Meckler called it “A picture of a picture of a picture of a picture.”. It works by using a ZKP to prove the knowledge of a ZKP, which proves the knowledge of a ZKP, etc. This effectively leads to a constant-size blockchain which can be verified by anyone easily, not like in many existing public blockchains where the more users a blockchain has, the more difficult it gets for the average user to verify. Coda does, interestingly, not use ZKP at all for privacy. Yet.
We’ve seen a few examples to intuit what ZKP means and why they are interesting to apply in blockchain technology. It is part of what we are working on at Datarella, implementing industrial blockchain solutions for clients and in RAAY. If you would like to dig deeper into some of the topics we’ve learned about today, here are some resources:
– https://jeremykun.com/2016/08/01/zero-knowledge-proofs-for-np/
– https://z.cash/technology/
– https://www.youtube.com/watch?v=DfEG5nhMRyQ&list=PLj80z0cJm8QHvg1ydi6rTEUK1SpxbtKnM
– https://medium.com/@VitalikButerin/zk-snarks-under-the-hood-b33151a013f6
– https://www.uow.edu.au/~bmaloney/wuct121/GraphsWeek10Lecture2.pdf
– https://zokrates.github.io/
– https://medium.com/aztec-protocol/how-to-code-your-own-confidential-token-on-ethereum-4a8c045c8651
by Michael Reuter | 17 February 2016 | Blockchain
With this post we do something quite unusual: we re-publish Apple’s letter to their customers. It’s not that we were such Apple fans that we would join the company’s marketing efforts but the encryption of iPhones has been discussed for a long time. And this week, the FBI and the United States government, demanded that Apple builds a new version of iOS bypassing security and creating a backdoor. This might not be the first case of such a request a private company being asked to install a backdoor (we know that some have done it) but Apple does not retreat. Please read this letter and add your thoughts to that topic – either here at Datarella or other channels – we think that this needs to be discussed in public. And – maybe – there even are (secure) technical solutions to comply with the FBI’s requests AND privacy.
“A message to our customers“, published by Apple, Inc., February 16:
February 16, 2016
A Message to Our Customers
The United States government has demanded that Apple take an unprecedented step which threatens the security of our customers. We oppose this order, which has implications far beyond the legal case at hand.
This moment calls for public discussion, and we want our customers and people around the country to understand what is at stake.
The Need for Encryption
Smartphones, led by iPhone, have become an essential part of our lives. People use them to store an incredible amount of personal information, from our private conversations to our photos, our music, our notes, our calendars and contacts, our financial information and health data, even where we have been and where we are going.
All that information needs to be protected from hackers and criminals who want to access it, steal it, and use it without our knowledge or permission. Customers expect Apple and other technology companies to do everything in our power to protect their personal information, and at Apple we are deeply committed to safeguarding their data.
Compromising the security of our personal information can ultimately put our personal safety at risk. That is why encryption has become so important to all of us.
For many years, we have used encryption to protect our customers’ personal data because we believe it’s the only way to keep their information safe. We have even put that data out of our own reach, because we believe the contents of your iPhone are none of our business.
The San Bernardino Case
We were shocked and outraged by the deadly act of terrorism in San Bernardino last December. We mourn the loss of life and want justice for all those whose lives were affected. The FBI asked us for help in the days following the attack, and we have worked hard to support the government’s efforts to solve this horrible crime. We have no sympathy for terrorists.
When the FBI has requested data that’s in our possession, we have provided it. Apple complies with valid subpoenas and search warrants, as we have in the San Bernardino case. We have also made Apple engineers available to advise the FBI, and we’ve offered our best ideas on a number of investigative options at their disposal.
We have great respect for the professionals at the FBI, and we believe their intentions are good. Up to this point, we have done everything that is both within our power and within the law to help them. But now the U.S. government has asked us for something we simply do not have, and something we consider too dangerous to create. They have asked us to build a backdoor to the iPhone.
Specifically, the FBI wants us to make a new version of the iPhone operating system, circumventing several important security features, and install it on an iPhone recovered during the investigation. In the wrong hands, this software — which does not exist today — would have the potential to unlock any iPhone in someone’s physical possession.
The FBI may use different words to describe this tool, but make no mistake: Building a version of iOS that bypasses security in this way would undeniably create a backdoor. And while the government may argue that its use would be limited to this case, there is no way to guarantee such control.
The Threat to Data Security
Some would argue that building a backdoor for just one iPhone is a simple, clean-cut solution. But it ignores both the basics of digital security and the significance of what the government is demanding in this case.
In today’s digital world, the “key” to an encrypted system is a piece of information that unlocks the data, and it is only as secure as the protections around it. Once the information is known, or a way to bypass the code is revealed, the encryption can be defeated by anyone with that knowledge.
The government suggests this tool could only be used once, on one phone. But that’s simply not true. Once created, the technique could be used over and over again, on any number of devices. In the physical world, it would be the equivalent of a master key, capable of opening hundreds of millions of locks — from restaurants and banks to stores and homes. No reasonable person would find that acceptable.
The government is asking Apple to hack our own users and undermine decades of security advancements that protect our customers — including tens of millions of American citizens — from sophisticated hackers and cybercriminals. The same engineers who built strong encryption into the iPhone to protect our users would, ironically, be ordered to weaken those protections and make our users less safe.
We can find no precedent for an American company being forced to expose its customers to a greater risk of attack. For years, cryptologists and national security experts have been warning against weakening encryption. Doing so would hurt only the well-meaning and law-abiding citizens who rely on companies like Apple to protect their data. Criminals and bad actors will still encrypt, using tools that are readily available to them.
A Dangerous Precedent
Rather than asking for legislative action through Congress, the FBI is proposing an unprecedented use of the All Writs Act of 1789 to justify an expansion of its authority.
The government would have us remove security features and add new capabilities to the operating system, allowing a passcode to be input electronically. This would make it easier to unlock an iPhone by “brute force,” trying thousands or millions of combinations with the speed of a modern computer.
The implications of the government’s demands are chilling. If the government can use the All Writs Act to make it easier to unlock your iPhone, it would have the power to reach into anyone’s device to capture their data. The government could extend this breach of privacy and demand that Apple build surveillance software to intercept your messages, access your health records or financial data, track your location, or even access your phone’s microphone or camera without your knowledge.
Opposing this order is not something we take lightly. We feel we must speak up in the face of what we see as an overreach by the U.S. government.
We are challenging the FBI’s demands with the deepest respect for American democracy and a love of our country. We believe it would be in the best interest of everyone to step back and consider the implications.
While we believe the FBI’s intentions are good, it would be wrong for the government to force us to build a backdoor into our products. And ultimately, we fear that this demand would undermine the very freedoms and liberty our government is meant to protect.
Tim Cook
