readme: update information on why specs

[tynes]

Description

We need a clear explanation about why specs and how to think about them.
Opening this PR for feedback.

[clabby]

Maybe my own preference, but I never quite liked the heavy use of design docs, barring when there was more of a research problem to answer. Design docs feel great when an idea of the feature is still being shaped, but I do prefer specs that end up with a concrete "why" on top of the "how."

Creating a spec first, esp. with us leaning into having multiple implementations, allows for more people to think about the feature through the lens of how they'd implement it. IMO this makes it easier to think of the benefits & challenges to push important discussion forwards, rather than focusing too much on "it looks great on paper."

[axelKingsley]

To counter @clabby a bit -- I think the design-doc phase is important because it ensures the spec has at least one strong implementation. Our design docs tend to discuss the properties of a good solution, and the limitations/boundaries within the solution space, and without that backdrop it feels easy to come up with specs that don't fit the real world.

If you spell out in specification what a Node must do, but doesn't describe any implementation, it is possible there is no implementation possible for a given feature. And even in cases where the implementation is technically possible, the tradeoffs when adding it to an already complex system could make it infeasible in production. Exploring those tradeoffs and ensuring the spec is achievable is important, but is not part of the spec itself.

As a toy example, we could say "Nodes must validate that all transactions are purple when considering a Block as valid". Sounds great on paper, now you have purple blocks. But if the computation to check transaction color is expensive, maybe you can only process a fraction of the transactions you used to (let alone the complexity to fault-prove it). And maybe purple-ness isn't even the end goal. A design doc can spell out the need, the potential solutions, and the feasibility of the proposed solution before it shows up as a specification to be followed.

[ajsutton]

I think I land somewhere in the middle. There are times when design docs are useful but the majority of our work shouldn't need one and we don't want to spend all our time writing docs just to follow "the process", especially docs like design docs which quickly become out of date and are no longer useful (or even misleading). The spec is the living document that will be maintained over time. So I think it needs to include some "why" but it is a reasonably light explanation - eg "the sequencing window exists to ensure L1 censorship resistance is inherited" but not "censorship resistance is an important property to give users confidence in the ability to withdraw their funds". That kind of technical why helps people implementing the spec to understand the details by setting their frame of reference and future spec changes by documenting key properties we want to preserve even if they're not strictly part of the implementation.

I'd also say that our experience shows that neither design docs, nor specs are good at ensuring the design actually works and can be implemented. I can't think of the last time a design doc or spec survived the first encounter with real world implementation.

So when we talk about doing specs first, we can't afford for that to mean doing waterfall development where we do a high level design (design doc), low level design (spec) and then implement. The software industry has consistently shown that approach doesn't work. We need to be doing design, spec and implementation in parallel so we benefit from the feedback loop that actually building an implementation and testing it provides.

[maurelian]

Suggestion for getting past this:

Suggested change

	- Include a light description of the problem that is being solved and the rationale but the majority of this should be recorded during the design doc phase
	- Include a light description of the problem that is being solved and the rationale

[mds1]

Include a light description of the problem that is being solved and the rationale but the majority of this should be recorded during the design doc phase

From past conversations, my understanding of design docs is that they are not intended to be deliverables themselves, and they should typically result in some other upstream change. For example, in ethereum-optimism/design-docs#21 we agreed on a new way to version and release L1 contracts, and we upstreamed it to the monorepo so it's more visible to contributors.

Therefore, I agree the problem and rationale should be recorded during the design doc phase, but I also think that info should still be upstreamed to the specs repo, to act as the permanent, single source of truth. It would provide a worse UX if readers had to jump to design docs—which are likely to become stale as specs are tweaked—to fully understand a spec

[mds1]

I think we should have more emphasis on the idea that specs should:

• Focus on describing the interfaces, behaviors, requirements, and safety conditions (e.g. when to error)
• Avoid mentioning implementation details and executable code snippets (e.g. interface code snippets are ok, implementation snippets are not).

This is an important distinction and leaking implementation details is an easy, expected mistake to make for people new to the spec-writing process. Quoting from a reply I left in #233 (comment):

To expand a bit so I don't think we've discussed this much, the rationale for "not leaking implementation details into the spec" is:

• Simplify spec maintenance: By specifying as little as possible to implement the requirements, we make it easier to keep the spec up to date by having less code to maintain. The code in this spec is not tested or audited, and we may end up tweaking it.
• Readability: The less code, the easier it is for a less-technical reader to understand the spec and ensure it's sound.
• Testing: The implementer of the spec has flexibility around the final result, as long as it meets the requirements. When implementing the spec, we write tests for all the requirements in here, but we wouldn't write a test to e.g. ensure we're using assembly since that's an implementation detail and not a requirement.

[maurelian]

I agree with this point. Would be good to see future PRs add this kind of content.

[ajsutton]

I think I land somewhere in the middle. There are times when design docs are useful but the majority of our work shouldn't need one and we don't want to spend all our time writing docs just to follow "the process", especially docs like design docs which quickly become out of date and are no longer useful (or even misleading). The spec is the living document that will be maintained over time. So I think it needs to include some "why" but it is a reasonably light explanation - eg "the sequencing window exists to ensure L1 censorship resistance is inherited" but not "censorship resistance is an important property to give users confidence in the ability to withdraw their funds". That kind of technical why helps people implementing the spec to understand the details by setting their frame of reference and future spec changes by documenting key properties we want to preserve even if they're not strictly part of the implementation.

I'd also say that our experience shows that neither design docs, nor specs are good at ensuring the design actually works and can be implemented. I can't think of the last time a design doc or spec survived the first encounter with real world implementation.

So when we talk about doing specs first, we can't afford for that to mean doing waterfall development where we do a high level design (design doc), low level design (spec) and then implement. The software industry has consistently shown that approach doesn't work. We need to be doing design, spec and implementation in parallel so we benefit from the feedback loop that actually building an implementation and testing it provides.

[ajsutton]

I don't think this is true and is actually an anti pattern. We shouldn't write the spec in isolation of implementation because we need the feedback cycle from actually implementing and testing it to know the spec actually works. It's extremely unusual for a good spec to be written without at least a proof of concept implementation being built alongside it. Even with the multi-client environment of L1, EIPs almost always have some kind of reference implementation or proof of concept that was used to work through the issues and get to an initial EIP.

[maurelian]

I also agree with this. For example I keep finding myself returning to the OPCM upgrades spec I've written, as I learn more about the problem space.

Two possible ways of addressing this:

1. Don't say it at all, the other two points should be sufficient to justify the existence of a spec:

Suggested change

- Ensures that we know what we are building before we build it

2. Acknowledge the dialectical interplay between the spec and the implementation:

Suggested change

	- Ensures that we know what we are building before we build it
	- During the building process:
	- Provides a venue for expressing a plain english view of the solution's properties. The spec is expected to evolve during this process, but must be accurate once it enters production.
	- After the building process:
	- Enables multiple implementations of the solution

[maurelian]

It looks like this tiny PR was held up by philosophical debates, but I think we'd be better off with imperfect specs guidance than none at all, and the right way to correct any imperfections would be through future PRs.

What I propose is:

• The Goals of the specs repo README are to clarify
  • The structure of the specs repo
  • what makes a good Spec doc.
  • what makes a bad Spec doc
  • How the specs repo should evolve alongside the #Protocol
• Non-goals for the specs repo README
  • not trying to figure out how design docs fit in

It looks to me like the "design docs" content was the main focus of debate here. If instead we just focus on what does/does not go into this repo without worrying about other processes, then I suspect other processes will be better able to shape themselves around it.

[maurelian]

IMO we should move this content into the design-docs repo.

Suggested change

	What is the difference between the specs and the [design docs](https://github.com/ethereum-optimism/design-docs)?
	- The design docs describe why and the specs describe how.
	- The design doc phase should be complete before the specs process is complete