Skip to content

Commit

Permalink
Preparation for TOSCA 2.0
Browse files Browse the repository at this point in the history
  • Loading branch information
tliron committed Aug 14, 2024
1 parent 02a5f6d commit ea44905
Show file tree
Hide file tree
Showing 83 changed files with 2,000 additions and 109 deletions.
8 changes: 4 additions & 4 deletions FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ I know, right? Now imagine writing a parser for it... Not only is it a complex l
Please join [OASIS's TOSCA community](https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca)
to help improve the language!

Meanwhile, Puccini includes [examples](examples/tosca/) of TOSCA's grammatical features with some
Meanwhile, Puccini includes [examples](examples/) of TOSCA's grammatical features with some
running commentary. Treat them as your playground. Also, if you have 4 hours to spare, grab some
snacks, get comfortable, and watch the author's free online course for TOSCA 1.0:
[part 1](https://www.youtube.com/watch?v=aMkqLI6o-58),
Expand Down Expand Up @@ -115,7 +115,7 @@ processor in Python, Ruby, etc., to do exactly what you need, e.g.:
Also check out [yq](https://mikefarah.gitbook.io/yq/), a great little tool for extracting YAML and
even performing simple manipulations. Example:

puccini-tosca compile examples/tosca/requirements-and-capabilities.yaml | yq '.vertexes.[]|select(.properties.name=="light6")'
puccini-tosca compile examples/1.3/requirements-and-capabilities.yaml | yq '.vertexes.[]|select(.properties.name=="light6")'

### Can I use text templating instead of TOSCA functions like `get_input`?

Expand Down Expand Up @@ -147,8 +147,8 @@ requiring Jinja2 template processing, after which the `.j2` extension would be s
TOSCA has a feature called "substitution mapping", which is useful for modeling service composition.
It is, however, a *design* feature. The implementation is up to your orchestration toolchain. See
our examples
[here](examples/tosca/substitution-mapping.yaml) and
[here](examples/tosca/substitution-mapping-client.yaml).
[here](examples/1.3/substitution-mapping.yaml) and
[here](examples/1.3/substitution-mapping-client.yaml).

Puccini intentionally does *not* support service composition. Each Clout file is its own graph
universe. If you need to create graph edges between vertexes in one Clout file and vertexes in other
Expand Down
50 changes: 25 additions & 25 deletions TUTORIAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Basic Usage

Let's start by compiling a self-contained local file:

puccini-tosca compile examples/tosca/descriptions.yaml
puccini-tosca compile examples/1.3/descriptions.yaml

What if the file imports other files? TOSCA `imports` can refer to either absolute
URLs or relative URLs ([RFC 1808](https://tools.ietf.org/html/rfc1808)). Note that
Expand Down Expand Up @@ -47,7 +47,7 @@ or wrap it in single quotes:

Puccini can also compile YAML from stdin:

cat examples/tosca/descriptions.yaml | puccini-tosca compile
cat examples/1.3/descriptions.yaml | puccini-tosca compile

Be aware that a stdin source does not have a path and thus cannot support relative
URLs.
Expand Down Expand Up @@ -76,45 +76,45 @@ The default output format is YAML but other formats are supported: JSON (and
[ARD](https://github.com/tliron/kutil/tree/master/ard/)-compatible extended JSON), XML,
CBOR, and MessagePack. Here's ARD-compatible XJSON:

puccini-tosca compile examples/tosca/descriptions.yaml --format=xjson
puccini-tosca compile examples/1.3/descriptions.yaml --format=xjson

By default the output is nicely indented and and colorized for human readability. You can
turn off prettification if you're interested in the most compact output:

puccini-tosca compile examples/tosca/descriptions.yaml --pretty=false
puccini-tosca compile examples/1.3/descriptions.yaml --pretty=false

Note that colorization will *always* be disabled in contexts that do not support it. In
other words it will likely only appear in stdout for terminal emulators that support ANSI
color codes. However, you can also specifically turn off colorization:

puccini-tosca compile examples/tosca/descriptions.yaml --colorize=false
puccini-tosca compile examples/1.3/descriptions.yaml --colorize=false

By default the output is sent to stdout but you can also send it to a file (without
colorization):

puccini-tosca compile examples/tosca/descriptions.yaml --output=clout.yaml
puccini-tosca compile examples/1.3/descriptions.yaml --output=clout.yaml

Of course if running in a shell you can also redirect stdout to a file (again, without
colorization):

puccini-tosca compile examples/tosca/descriptions.yaml > clout.yaml
puccini-tosca compile examples/1.3/descriptions.yaml > clout.yaml

You can increase the verbosity of logging using `-v` or even `-vv`:

puccini-tosca compile examples/tosca/descriptions.yaml -vv
puccini-tosca compile examples/1.3/descriptions.yaml -vv

By default all the log messages go to stderr but we can send them to a file:

puccini-tosca compile examples/tosca/descriptions.yaml -vv --log=puccini.log
puccini-tosca compile examples/1.3/descriptions.yaml -vv --log=puccini.log
cat puccini.log

If you only want to see the logs and not the Clout output:

puccini-tosca compile examples/tosca/descriptions.yaml -vv > /dev/null
puccini-tosca compile examples/1.3/descriptions.yaml -vv > /dev/null

To suppress all output (if you're only interested in the return error code):

puccini-tosca compile examples/tosca/descriptions.yaml --quiet
puccini-tosca compile examples/1.3/descriptions.yaml --quiet

Also note that there is a `puccini-tosca parse` command that provides a lot
of internal diagnostic information about the language parser. It's generally
Expand All @@ -128,7 +128,7 @@ More on Compilation

Let's try to compile a TOSCA service template that requires inputs:

puccini-tosca compile examples/tosca/inputs-and-outputs.yaml
puccini-tosca compile examples/1.3/inputs-and-outputs.yaml

You'll see that Puccini reported a "problem" regarding the unassigned input. Any and all
compilation errors, whether they are syntactical, grammatical, or topological, are
Expand All @@ -138,11 +138,11 @@ problem reporting is one of its most powerful features.
By default problems are reported in a human-readable format. However, like the Clout
output, problems can be formatted for easier consumption by other tools:

puccini-tosca compile examples/tosca/inputs-and-outputs.yaml --problems-format=json
puccini-tosca compile examples/1.3/inputs-and-outputs.yaml --problems-format=json

Let's set that missing input:

puccini-tosca compile examples/tosca/inputs-and-outputs.yaml --input=ram=1gib
puccini-tosca compile examples/1.3/inputs-and-outputs.yaml --input=ram=1gib

In this case the input is a string (actually a TOSCA `scalar-unit.size`), but note that
the the input format is YAML, which is also JSON-compatible, so that complex input
Expand All @@ -152,14 +152,14 @@ you can use the `--input` flag more than once to provide multiple inputs.
Inputs can also be loaded from a file (locally or at a URL) as straightforward YAML:

echo 'ram: 1 gib' > inputs.yaml
puccini-tosca compile examples/tosca/inputs-and-outputs.yaml --inputs=inputs.yaml
puccini-tosca compile examples/1.3/inputs-and-outputs.yaml --inputs=inputs.yaml

By default the compiler will "resolve" the topology, meaning that it will atempt to satisfy
all node template requirements and create relationships, thus completing the graph. However,
sometimes it may be useful to disable the resolution phase in order to avoid excessive problem
reports:

puccini-tosca compile examples/tosca/requirements-and-capabilities.yaml --resolve=false
puccini-tosca compile examples/1.3/requirements-and-capabilities.yaml --resolve=false

When you turn off the resolution phase you will indeed see no relationships in the Clout
(you'll see that the `edgesOut` for all vertexes is an empty list).
Expand All @@ -180,14 +180,14 @@ from runtime resources.

You can see the call stubs by compiling this example:

puccini-tosca compile examples/tosca/functions.yaml
puccini-tosca compile examples/1.3/functions.yaml

You'll notice that the call stubs all have the special `$functionCall` key.

How do we call the functions? In Puccini we refer to this as "value coercion". As a
convenience we can use the `--coerce` flag to coerce the values during compilation:

puccini-tosca compile examples/tosca/functions.yaml --coerce
puccini-tosca compile examples/1.3/functions.yaml --coerce

You'll see that all properties now have their actual values rather than call stubs.

Expand All @@ -201,19 +201,19 @@ data type.

Let's try this example:

puccini-tosca compile examples/tosca/data-types.yaml --coerce
puccini-tosca compile examples/1.3/data-types.yaml --coerce

Now, edit `examples/tosca/data-types.yaml` and break a constraint. For example, the
Now, edit `examples/1.3/data-types.yaml` and break a constraint. For example, the
`constrained_string` property requires a minimum length of 2 and a maximum length of
5, so let's set its value to a string with length 6, `ABCDEF` (at line 267), and
compile and coerce again:

puccini-tosca compile examples/tosca/data-types.yaml --coerce
puccini-tosca compile examples/1.3/data-types.yaml --coerce

You'll see a problem reported telling you exactly which constraint failed and where.
Now, let's compile this same file without coercion (the default behavior):

puccini-tosca compile examples/tosca/data-types.yaml
puccini-tosca compile examples/1.3/data-types.yaml

The problem was not reported this time.

Expand All @@ -233,13 +233,13 @@ function and constraint call stubs are implemented.

Let's use the `puccini-clout` tool to list these embedded scriptlets:

puccini-tosca compile examples/tosca/requirements-and-capabilities.yaml --output=clout.yaml
puccini-tosca compile examples/1.3/requirements-and-capabilities.yaml --output=clout.yaml
puccini-clout scriptlet list clout.yaml

Note that `puccini-clout` can also accept Clout input from stdin, allowing us to pipe
the two tools:

puccini-tosca compile examples/tosca/requirements-and-capabilities.yaml | puccini-clout scriptlet list
puccini-tosca compile examples/1.3/requirements-and-capabilities.yaml | puccini-clout scriptlet list

Let's extract a scriptlet's source code:

Expand Down Expand Up @@ -270,7 +270,7 @@ the topology:
Note another shortcut for `puccini-tosca compile`: you can use the `--exec` flag to
execute scriptlets right after compilation, thus skipping the Clout intermediary:

puccini-tosca compile examples/tosca/requirements-and-capabilities.yaml --exec=assets/tosca/profiles/common/1.0/js/visualize.js
puccini-tosca compile examples/1.3/requirements-and-capabilities.yaml --exec=assets/tosca/profiles/common/1.0/js/visualize.js

See [here](executables/puccini-clout/) for more information about the `puccini-clout`
tool.
Expand Down
8 changes: 4 additions & 4 deletions clout/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ General implementation-specific properties for the whole topology.
The difference between `metadata` and `properties` is a matter of convention. Generally, `properties`
should be used for data that is implementation-specific while `metadata` should be used for tooling.
It is understood that this distinction might not always be clear and thus you should not treat the
two areas differently in terms of state management.
two areas differently in terms of state management.

### `vertexes` (map of string to Vertex)

Expand Down Expand Up @@ -254,7 +254,7 @@ It may also have the follow optional fields for debugging information:
* `typeMetadata`: a map of strings associated with the value's type
* `localMetadata`: a map of strings associated with the value itself

Example (generated from [this TOSCA example](../examples/tosca/data-types.yaml)):
Example (generated from [this TOSCA example](../examples/1.3/data-types.yaml)):

```yaml
lowercase_string_map:
Expand All @@ -271,7 +271,7 @@ lowercase_string_map:
name: tosca.function.concat
path: topology_template.node_templates["data"].properties["lowercase_string_map"]["concat:¶ - recip¶ - ient"]
row: 194
url: file:/Depot/Projects/RedHat/puccini/examples/tosca/data-types.yaml
url: file:/Depot/Projects/RedHat/puccini/examples/1.3/data-types.yaml
$primitive: Puccini
$meta:
type: map
Expand All @@ -286,7 +286,7 @@ lowercase_string_map:
name: tosca.constraint.pattern
path: topology_template.node_templates["data"].properties["lowercase_string_map"]
row: 194
url: file:/Depot/Projects/RedHat/puccini/examples/tosca/data-types.yaml
url: file:/Depot/Projects/RedHat/puccini/examples/1.3/data-types.yaml
value:
type: string
```
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
tosca_definitions_version: tosca_simple_yaml_1_3

# To apply the constraints use "--coerce":
# puccini-tosca compile --coerce examples/tosca/data-types.yaml
# puccini-tosca compile --coerce examples/1.3/data-types.yaml

metadata:

Expand Down Expand Up @@ -245,7 +245,7 @@ topology_template:
version: 1.2.3.beta-4

# Range is actually quite limited in use:
# * Unsigned integers only
# * Unsigned integers only
# * Upper must be >= than lower
# You can also use "UNBOUNDED" string for the upper bound, which will be converted to
# the maximum uint value
Expand All @@ -268,7 +268,7 @@ topology_template:
<entry>3</entry>
- |-
</tag2>
# Constraints: min length = 2 and max length = 5
# (The length-related constraints work on both strings and lists)
constrained_string: ABCDE
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ topology_template:
- ::1
# Some functions support special "modelable entity names", such as "SELF":
management_port: { get_property: [ SELF, admin_port ] }
# (Also see: examples/tosca/source-and-target.yaml)
# (Also see: examples/1.3/source-and-target.yaml)
capabilities:
ingress:
properties:
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ tosca_definitions_version: tosca_simple_yaml_1_3
# Note that by default "puccini-tosca compile" will attempt to resolve the topoloy
# (meaning: satisfy all requirements)
# But it's possible to disable it:
# puccini-tosca compile --resolve=false examples/tosca/requirements-and-capabilities.yaml
# puccini-tosca compile --resolve=false examples/1.3/requirements-and-capabilities.yaml

metadata:

Expand Down Expand Up @@ -170,7 +170,7 @@ topology_template:
properties:
- frequency:
greater_than: 100 # 100 hz

# You can add a relationship type to the requirement
# (If you don't, Puccini creates an empty relationship with no properties)
light9:
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
60 changes: 60 additions & 0 deletions examples/2.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
TOSCA Grammar Examples
======================

Here you'll find a nice little playground for learning the basic features of TOSCA.

Try modifying the files and then see if the service template can still be parsed.

Did you make a mistake? Puccini should catch all syntactical and grammatical errors and provide you
with a helpful report.

Syntax
------

* [Namespaces](namespaces.yaml)
* [Copy](copy.yaml)
* [DSL Definitions](dsl-definitions.yaml)
* [Unicode](unicode.yaml)

Data
----

* [Data Types](data-types.yaml)
* [Descriptions](descriptions.yaml)
* [Metadata](metadata.yaml)
* [Attributes](attributes.yaml)

Entities
--------

* [Artifacts](artifacts.yaml)
* [Policies and Groups](policies-and-groups.yaml)
* [Inputs and Outputs](inputs-and-outputs.yaml)

Topology
--------

* [Requirements and Capabilities](requirements-and-capabilities.yaml)

Functions
---------

* [Functions](functions.yaml)
* [Source and Target](source-and-target.yaml)

Orchestration
-------------

* [Interfaces](interfaces.yaml)
* [Workflows](workflows.yaml)

Composition
-----------

* [Substitution Mapping](substitution-mapping.yaml)
* [Substitution Mapping Client](substitution-mapping-client.yaml)

NFV
---

* [Simple for NFV](simple-for-nfv.yaml)
Loading

0 comments on commit ea44905

Please sign in to comment.