Against L1 Merkle Machines

Technical Discussion
1

Been thinking about the registry a lot this week. Merkle trees are a really elegant optimization but the more requirements we tack on to the registry the scarier the idea of using them seems.

It seems like we’re trying to re-solve some of the scaling issues other really smart people have worked on. Here are some examples:

-Manufacturers including duplicate chip PK hashes in merkles. We can rely on observant slashers to disincentivize this case but it still relies on multiple slashers being active to boost the perceived security of the system. (Sort of re-engineering Optimism’s scaling model. To be clear, this is different from the private-key slashers). The alternative to slashers is having each client verify the integrity of the merkles but this has it’s own problems:

-Verifier apps needing to download the entire merkle is no bueno as n devices grows. Alternatively, we can follow an RPC model where various RPCs verify registry integrity but then that’s re-engineering an existing scaling solution (RPCs)

-To my limited imagination, universal resolution of content from a physical chip either relies on flashing stuff to each individual chip before shipping them out to TSMs OR some “resolver permission handoff” done purely in software (Like an ENS domain owner handing off ownership similar to how one transfers an NFT). Manus can flash merkle paths onto the chips but that complicates manufacturing with a second step. It also complicates verifying. On the other hand, the software-only solution would be super tricky to accomplish with merkles and potentially open up security concerns. Permissioned registries (i.e. NFTs) like that used in a software-only solution are a huge push for the people building scaling solutions. So by crafting a custom solution using merkles we’re once again re-engineering their work.

-In a similar vein, we need to worry about cross chain, and cross-chain bridges for permissioned registries(NFTs) are something that other people are already working on. So why reinvent the wheel here too?

Besides re-engineering scaling solutions, there are some other reasons I think moving to L2 is the move:

-Potential to monetize registry for KONG’s benefit. We can charge a small amount (on the order of cents) per-chip if we create 1 registry entry for each one. This could let KONG directly benefit from other manus using the registry. I don’t think charging per-chip is feasible with merkle machines.

-GSD (getting shit done) We could grind out an L2 implementation pretty fast. No merkle magic, just a simple registry and some solution to “yank” the registry entries onto L1 at the expense of the user/TSM, leveraging existing oracle or NFT bridge technologies.

-The create2 contracts explained in @cadillion’s scheme can be deployed contracts viewable on etherscan (or scan) to simplify integration and communication with other devs. In the future we can actually use create2 to minimize gas costs.

-Communication becomes much easier. We basically say “this works like ENS but for physical assets” instead of having to explain how our merkle machines work. I suspect people (esp. big corps) will want to understand how the system works before risking their product line on its viability and security.

Future gas forecast:
Damn this is starting to feel like a DD post on r/WSB. I hate getting into speculation but I feel it’s necessary before making this decision. I’m neutral-to-bearish on gas costs in the long-term. Vitalik himself believes many cool apps are being priced out from ETH and that gas prices need to go down. So already, the demand is being “held back” by gas prices. As @cameron said, “open a bigger highway and more cars will flood in”, so the merge and other scaling solutions will definitely increase the supply of computation but at the same time increase demand since new apps will be built leveraging the lower fees. I see no reason to believe that the new equilibrium caused by the new supply and new demand levels will be significantly higher than it is now. Hopefully it’s lower. Hence my neutral-to-bearish long-term take on gas on L1.

If we go forward with the registry being written directly to either L1 (post merge) or L2 (now), I’m pretty sure we can get gas/chip down to ~2k (amortized) as Azuki has done with their excellent ERC721-A implementation:

That’s under $1 /chip on L1 with current gas prices! So forecasting that gas stays ~the same long-term on L1, that’s just barely non-economical IMO. So moving to L2 would almost certainly be a viable long-term solution that keeps per-chip economics low (<1/10 price of chip).

I completely understand the reluctance of @cameron from getting burned by gas fees again. I really respect his opinion, so I felt the need to write this long post to justify why I’m basically disagreeing with the move to stay on L1 using merkle machines.

2 Likes
2

Would this mean we would have to launch separate registries on polygon, arbitrum, optimism, etc. And the contents of each registry would differentiate between each L2?

3 Likes
3

Good question because tbh I haven’t fully thought through this. One scheme that I think could work is the owner of the registry entry is able to transfer read-only copies of registry entries to other chains. When the entry is on another chain, they aren’t allowed to edit the original entry. They can always transfer it back tho (unless they send to the 0 address while on the other chain). We could potentially use NFT bridges for this transferring stuff to cut down on dev time.

1 Like
4

I agree that –

  • Merkle trees are difficult to deal with for individually tagged items and
  • L2’s, efficiency improvements, etc., will likely allow for low cost 1-1 linking for a while

However, I think that there are some big problems being “L2 first” –

  • Which L2?
  • Where is the canonical “source of truth” for chips

This overall discussion is pushing me more and more towards the notion that ultimately our own chain makes the most sense. In the meantime I believe that a EIP-3668 approach might make the most sense. In this world:

  • Off-chain merkle trees are the source of truth for the initial creation of chips (as we’ve done for a while now).
  • The root of the registry lives on L1, and importantly, so does the resolution path.
  • Manufacturer merkle roots live directly on L1 simple because this mirrors the nature of chip manufacturing (e.g. you typically make many chips at once).
  • TSMs can live on L1 or L2; if they live on L2, all resolution calls still route through L1 thanks to EIP-3668 formatted offchain calls. The TSM will need to at a bare minimum register on L1 (think of this as being similar to registering an ENS domain) in order to point batches of chip roots to L2.
  • On L2, we can be agnostic to whether or not we use create2 or just deploy the contract.

I highly recommend watching Nick Johnson from ENS explain EIP-3668 – in the video he basically explains how what we’ve already done with chip Merkle trees and custom interfaces to review these Merkle trees is exactly what EIP-3668 is attempting to standardize and solve (notably Merkle proof fetching).

Benefits:

  • EIP-3668 support is already in Ethers 5.6.2+
  • As KONG we don’t need to make a bet on an L2 that we choose to support
  • Contract calls on L1 are aesthetically easy to deal with; we don’t need to explain that end customers need to worry about a niche L2 implementation
  • Broad adoption of EIP-3668 will obviate the need for us to explain our choices around Merkle trees as it “just works”
  • The Arweave work is a perfect fallback for chips that have not already been registered with a TSM; Arweave becomes another gateway here that can return a generic resolution path via an L1 call.

I don’t think charging per-chip is feasible with merkle machines.

It should be simple: the TSM that includes the root must also include a count of chips (each Merkle leaf). This should be verifiable and could act as a means of levying a fee for the protocol.

Communication becomes much easier. We basically say “this works like ENS but for physical assets” instead of having to explain how our merkle machines work. I suspect people (esp. big corps) will want to understand how the system works before risking their product line on its viability and security.

Given ENS is moving towards Durin, we can move away from “merkle machines” to “EIP-3668 conformant” which gives us the benefit of letting end users choose the L2 they want to live on.

I think that there are a few more pieces that I need to understand prior to implementing Durin, but I believe this is the rough outline:

  • Deploy a new L1 registry (our current registry with Merkle roots is non-upgradeable, so I believe that this new registry should become the canonical one going forward)
  • Ensure that the registry can handle “wrapping” a root to L2
  • Go wild on L2 :laughing: – e.g. a TSM can deploy its own contract here where chips are matched directly to contracts, assets, escrows, etc.
  • Adapt the HaLo scanner and KONG apps to Ethers 5.6.2 and making calls via the new registry to resolve chips.

Additional reading:

1 Like
5

In principle, I don’t think there’s any problem with relying on EIP-3668 while the manufacturer and the TSM are trusted entities. The point is for someone reading the contract ABI to see that the contract relies on off-chain data and to transparently see what urls they might be redirected to. Long term though, it probably makes sense to connect it to some kind of oracle service or an on-chain entity instead.

I agree that layer 2 does have a few long term challenges when thinking about merging registries across layers, but I also think the risk is low if you choose an EVM compatible L2 like optimism, which allows you to deploy both places with the same contracts. There’s some added concern in terms of L2 withdrawals guarantees and complicating things by requiring buyers to get to the L2, but in the short-term for getting something out the door, it could make sense.

If I were the one investing in a solution, I’d probably start on L1 with 3668, and put a copy on optimism for people who didn’t want to pay the gas fees, or who want small batch productions.

6

@ramicaza adding some resources here to fully demo the 3668 flow:

My sense is that Optimism would be easiest to support out of the box, however, we could also create our own simple bridging server until we could point directly to Polygon nodes to respond to the RPC calls.

1 Like
7

Appreciate the well thought out replies, EIP-3668 seems like a great tool.

After following the discussion in the discord, I have a crazy smoothbrained alternative to consider. It’s so simple I don’t think it will add any further “mental strain” to the conversation.

  1. Define a clear interface that we will be exposing to “the world” via the root registry which always lives on L1.

Maybe something like resolveChip(public key hash) → (address, chain id, init code hash)
Or more similar to ens:
resolveChip(pk hash) → (registrant, controller, text records, etc.)

We should really nail down this interface so that it never needs to be “upgraded” or changed in the future.

  1. Use 3668 to resolve data from an L2 (probably optimism)

  2. Now we have a registry with which we can implement complex permission handoffs and other constraints on in L2 but is always accessible via the nice interface in L1. Even if we want to change to merkle machines in the future the interface could abstract that change away from contracts and dapps that use the registry.

This should not be a problem because if we want to switch L2s we can keep the interface but swap the underlying L2. Re-entering all the entries would probably be “cheap enough” based on my WSB style gas forecast.

This would now be the L1 interface.

I believe all the benefits your listed would be retained in this new scheme minus the ARweave fallback.

I think we get the best of both of these worlds. Contracts on optimism can use the registry directly while contracts on L1 can call the interface. For contracts on any other L2 or even other chain we can create (or use an existing) bridge using some consortium of signers down the line.

1 Like
8

I like this smooth brain approach to start with; it does, however, limit other manufacturers and implies trust in a single path.

I am debating how we can quickly combine parts of the @cadillion manufacturer<>TSM<>chip Merkle structure for a v0, then build out the intervening pieces.

Adding some color from Discord from @cadillion:

To summarize what I’m hearing:

  • a manufacturer namespace should resolve all chips in its final merkle and should keep track of whether the chip TSM is writable or immutable, and sign the header with a Schnorr signature including the chip’s public key, the root public key, and the manufacturer public key. 3668 can store the hashes or urls of associated TSMs.

  • a TSM namespace should resolve all content in its final merkle and should keep track of the metadata associated with the Schnorr signature provided by the manufacturer, eg registrant/controller/registration date/expiration date/resolver/text records as in ENS, producing a new Schnorr signature containing TSM and the previous manufacturer signature. The text record can be replaced or combined with 3668.

  • a chip contains either the link to the manufacturer registry or the hash of the create2 initialization code, by which it can look up its associated TSMs. If desired, an NFT representing the chip’s namespace can be sent to the create2 address to provide more flexibility in the timing of connecting a chip to its content, or to connect the chip to an end user address.

With the Schnorr signatures, a contract can verify all four participants are valid (root, manufacturer, TSM, chip) without redundantly encoding chips across TSMs.

Stepping back, I think it’s important to reiterate the goals:

  1. Link chips to content, contracts on an individual or ‘batch’ basis
  2. Allow an end use to trace the nature of a chip from manufacturer <> TSM <> owner
  3. Specify OCAPs relative to chips*

*I think @cadillion you have a stronger sense of this being an important goal within the scope of the registry than @ramicaza or myself. My take right now is that chips are largely immutable and that the way in which assets use signatures from the chip could be considered to be outside of our purview. This, of course, opens up risks of TSMs doing bad things.

One can envision a clone of ENS which carries out these functions:

  • kong is the root, anyone can register *.kong as a manufacturer
  • arx.kong is a manufacturer which adds 1,000’s of chip records to the domain (ignoring merkles for the moment). Anyone can register *.arx.kong as a TSM.
  • metafactory.arx.kong is a TSM which adds 1,000’s of chip records to the domain where the chip public key hash is the key and the create2 init code or deployed contract is the value.

This registry would satisfy goal number 1, but not goals 2 or 3 as the TSM could add or modify arbitrary records (again, if we just cloned ENS).

Here is where the header with its series of public keys matter; it acts as a certificate such that a chip missing the manufacturer public key would fail to certify. Likewise, a bare minimum OCAP structure is required (even if it’s just by implication “records are write once, immutable”).

Circling back to the v0 implementation, I am wondering if there is a simplified version wherein we can satisfy goals 1 & 2 while placing trust in the TSM.

1 Like
9

I think you’ve accurately captured my thoughts here, and the smooth brain approach still feels extensible. I think the main thing is just to reference an external upgradeable contract for each of the major functions we anticipate, and to implement the trivial case for each of them.

To elaborate on OCAP, there’s a number of cool interactions that are enabled by having OCAP extensions:

  • A TSM can add OCAPs to arbitrary addresses such that a whitelist can unlock chip functionality
    • a coliving networks doorlocks can whitelist pre-existing Halo tokens
    • a list of halos can withdraw from a shared contract wallet
    • certain halos in a batch have view vs edit permissions
  • OCAPs allow clients to burn trust with certain providers
    • a TSM maliciously lists my halo on a subdomain, I can add that subdomain to a denylist on my halo’s record
  • Importantly OCAPs create flexibility in terms of off-chain behavior
    • What happens when a chip is lost in transit between manufacturer and TSM or TSM and send user? Can we give special stakes or OCAPs to distributors as part of the escrow chain? What’s the dispute resolution process?
    • What happens if a chip is maliciously separated from the object it vouches for and attached to a fake? Is there a trusted assessor who has OCAP?
  • In a world where halos are ubiquitous and cheap, can they be attached to IoT and sign messages for device-based oracles?
    • OCAPs can now coordinate rules of how IoT devices trade data and read/write access with each other

For me there are three distinct use-cases:

  • Halo is persistently attached to an object identity
  • Halo is persistently attached to a human identity and carries multiple object identities
  • Halo is disposable as a short-term proof of escrow

Let me know if you see that differently.

10

This really helps, thanks, notably thinking about chips relative to ideas like this:

  • a coliving networks doorlocks can whitelist pre-existing Halo tokens
  • a list of halos can withdraw from a shared contract wallet
  • certain halos in a batch have view vs edit permissions

I think the key smooth brain piece we’re trying to resolve now is resolution that is one-one vs. one-many on the chip/TSM side.

  • One-one seems immediately tractable, resolves immediate blockers for business cases and is super simple (the chip can only exist as a single entity).
  • One-many (i.e. multiple TSMs can include the same chip across their registries) seems to introduce some logistical hurdles in cases where a link is intended to be deterministic by the first TSM. BUT I love the idea that this could really empower chips.

Maybe there is a simple solution here – burn on TSM on the chip? Or create a “primary” relationship in the first manufacturer <> TSM link?

1 Like
11

I think the simple solution works, it just requires you to know the intended TSM before the manufacturer ships the chip, which AFAIK is accurate for v1? A simple solution to the simple case.

In the future it may be desirable for a manufacturer to ship to a reseller and add the TSM on resale.

The advanced solution is to require both the TSM and the chip to sign the TSM’s content merkle entry, so that way any fake TSM won’t have a valid signature from the chip and any application can verify the validity of the TSM as long as the chip is present. The Schnorr signature really makes sense for this application, as any fake TSM won’t be able to provide a valid cosign.

That being said, I don’t think the complexity of multisig schema is necessary for v1.