Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

readme: update information on why specs #349

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

readme: update information on why specs #349

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,23 @@ Please chat with us on the [discussion board](https://github.com/ethereum-optimi

We welcome your contributions. Read through [CONTRIBUTING.md](./CONTRIBUTING.md) for a general overview of the contributing process for this repository.

### How to Think about Specs

The specs process is a schelling point for where core developers go to modify the OP Stack. A spec serves multiple purposes:
- Ensures that we know what we are building before we build it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is true and is actually an anti pattern. We shouldn't write the spec in isolation of implementation because we need the feedback cycle from actually implementing and testing it to know the spec actually works. It's extremely unusual for a good spec to be written without at least a proof of concept implementation being built alongside it. Even with the multi-client environment of L1, EIPs almost always have some kind of reference implementation or proof of concept that was used to work through the issues and get to an initial EIP.

Copy link
Contributor

@maurelian maurelian Dec 23, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also agree with this. For example I keep finding myself returning to the OPCM upgrades spec I've written, as I learn more about the problem space.

Two possible ways of addressing this:

  1. Don't say it at all, the other two points should be sufficient to justify the existence of a spec:
Suggested change
- Ensures that we know what we are building before we build it
  1. Acknowledge the dialectical interplay between the spec and the implementation:
Suggested change
- Ensures that we know what we are building before we build it
- During the building process:
- Provides a venue for expressing a plain english view of the solution's properties. The spec is expected to evolve during this process, but must be accurate once it enters production.
- After the building process:
- Enables multiple implementations of the solution

- Helps code reviewers and auditors understand what the behavior is supposed to be
- Enables multiple implementations

Some guidelines for writing a good spec:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have more emphasis on the idea that specs should:

  • Focus on describing the interfaces, behaviors, requirements, and safety conditions (e.g. when to error)
  • Avoid mentioning implementation details and executable code snippets (e.g. interface code snippets are ok, implementation snippets are not).

This is an important distinction and leaking implementation details is an easy, expected mistake to make for people new to the spec-writing process. Quoting from a reply I left in #233 (comment):

To expand a bit so I don't think we've discussed this much, the rationale for "not leaking implementation details into the spec" is:

  • Simplify spec maintenance: By specifying as little as possible to implement the requirements, we make it easier to keep the spec up to date by having less code to maintain. The code in this spec is not tested or audited, and we may end up tweaking it.
  • Readability: The less code, the easier it is for a less-technical reader to understand the spec and ensure it's sound.
  • Testing: The implementer of the spec has flexibility around the final result, as long as it meets the requirements. When implementing the spec, we write tests for all the requirements in here, but we wouldn't write a test to e.g. ensure we're using assembly since that's an implementation detail and not a requirement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this point. Would be good to see future PRs add this kind of content.

- Thoroughly describe the interfaces that other projects may interact with
- Include a light description of the problem that is being solved and the rationale but the majority of this should be recorded during the design doc phase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe my own preference, but I never quite liked the heavy use of design docs, barring when there was more of a research problem to answer. Design docs feel great when an idea of the feature is still being shaped, but I do prefer specs that end up with a concrete "why" on top of the "how."

Creating a spec first, esp. with us leaning into having multiple implementations, allows for more people to think about the feature through the lens of how they'd implement it. IMO this makes it easier to think of the benefits & challenges to push important discussion forwards, rather than focusing too much on "it looks great on paper."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To counter @clabby a bit -- I think the design-doc phase is important because it ensures the spec has at least one strong implementation. Our design docs tend to discuss the properties of a good solution, and the limitations/boundaries within the solution space, and without that backdrop it feels easy to come up with specs that don't fit the real world.

If you spell out in specification what a Node must do, but doesn't describe any implementation, it is possible there is no implementation possible for a given feature. And even in cases where the implementation is technically possible, the tradeoffs when adding it to an already complex system could make it infeasible in production. Exploring those tradeoffs and ensuring the spec is achievable is important, but is not part of the spec itself.

As a toy example, we could say "Nodes must validate that all transactions are purple when considering a Block as valid". Sounds great on paper, now you have purple blocks. But if the computation to check transaction color is expensive, maybe you can only process a fraction of the transactions you used to (let alone the complexity to fault-prove it). And maybe purple-ness isn't even the end goal. A design doc can spell out the need, the potential solutions, and the feasibility of the proposed solution before it shows up as a specification to be followed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I land somewhere in the middle. There are times when design docs are useful but the majority of our work shouldn't need one and we don't want to spend all our time writing docs just to follow "the process", especially docs like design docs which quickly become out of date and are no longer useful (or even misleading). The spec is the living document that will be maintained over time. So I think it needs to include some "why" but it is a reasonably light explanation - eg "the sequencing window exists to ensure L1 censorship resistance is inherited" but not "censorship resistance is an important property to give users confidence in the ability to withdraw their funds". That kind of technical why helps people implementing the spec to understand the details by setting their frame of reference and future spec changes by documenting key properties we want to preserve even if they're not strictly part of the implementation.

I'd also say that our experience shows that neither design docs, nor specs are good at ensuring the design actually works and can be implemented. I can't think of the last time a design doc or spec survived the first encounter with real world implementation.

So when we talk about doing specs first, we can't afford for that to mean doing waterfall development where we do a high level design (design doc), low level design (spec) and then implement. The software industry has consistently shown that approach doesn't work. We need to be doing design, spec and implementation in parallel so we benefit from the feedback loop that actually building an implementation and testing it provides.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion for getting past this:

Suggested change
- Include a light description of the problem that is being solved and the rationale but the majority of this should be recorded during the design doc phase
- Include a light description of the problem that is being solved and the rationale

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include a light description of the problem that is being solved and the rationale but the majority of this should be recorded during the design doc phase

From past conversations, my understanding of design docs is that they are not intended to be deliverables themselves, and they should typically result in some other upstream change. For example, in ethereum-optimism/design-docs#21 we agreed on a new way to version and release L1 contracts, and we upstreamed it to the monorepo so it's more visible to contributors.

Therefore, I agree the problem and rationale should be recorded during the design doc phase, but I also think that info should still be upstreamed to the specs repo, to act as the permanent, single source of truth. It would provide a worse UX if readers had to jump to design docs—which are likely to become stale as specs are tweaked—to fully understand a spec

- Describe things in terms of state machines and invariants
- It should be clear that it solves the problem at hand

What is the difference between the specs and the [design docs](https://github.com/ethereum-optimism/design-docs)?
- The design docs describe why and the specs describe how.
- The design doc phase should be complete before the specs process is complete
Comment on lines +33 to +35
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should move this content into the design-docs repo.

Suggested change
What is the difference between the specs and the [design docs](https://github.com/ethereum-optimism/design-docs)?
- The design docs describe why and the specs describe how.
- The design doc phase should be complete before the specs process is complete


### Dependencies

**Rust Toolchain**
Expand Down