Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Evaluate possible use of diagrams package (replace mermaid) #127

Open
juztas opened this issue Jul 24, 2024 · 10 comments
Open

Evaluate possible use of diagrams package (replace mermaid) #127

juztas opened this issue Jul 24, 2024 · 10 comments
Assignees
@sunami09
Labels
v0.2
Milestone
RTMon v0.2

Comments

@juztas
Copy link
Collaborator

juztas commented Jul 24, 2024

Mermaid is nice, but there is complexity on topology generation and overlapping items (in case big topo). There is diagrams py package - which can generate image and we could show it. Something to try out and see if that gives nicer output.
https://diagrams.mingrammer.com/docs/getting-started/examples
@juztas juztas added the v0.2 label Jul 24, 2024
@juztas
Copy link
Collaborator Author

juztas commented Aug 14, 2024

Example run uploaded here: #130

@juztas
Copy link
Collaborator Author

juztas commented Aug 15, 2024

Here is what needs to be implemented:

  • In addition to mermaid - it should also generate diagrams graph and store image on a directory, e.g. /srv/images/
  • http endpoint should allow unauthenticated access to get images (will be needed for next step)
  • grafana html panel template - which loads specified image in grafana dashboard

We would have it running in parallel for few shots and compare how it looks between mermaid and diagrams

@sunami09
Copy link
Collaborator 

manifest = {
    "Ports": [
      {
        "Site": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_UCSD_OASIS:oasis",
        "Peer": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502",
        "Vlan": "3603",
        "Mac": "?port_mac?",
        "Name": "Pc502"
      },
      {
        "Site": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_UCSD_OASIS:oasis",
        "Peer": "urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500",
        "Vlan": "3603",
        "Mac": "?port_mac?",
        "Name": "Pc500"
      },
      {
        "Site": "urn:ogf:network:nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:Ethernet32",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_SDSC:edgecore_s0",
        "Peer": "?peer?",
        "Host": [
          {
            "IPv6": "?ipv6?",
            "IPv4": "10.251.87.82/30",
            "Interface": "enp168s0np0",
            "Mac": "a0:88:c2:86:ee:7c",
            "Name": "T2_US_SDSC:k8s-gen5-01.sdsc.optiputer.net"
          }
        ],
        "Vlan": "3603",
        "Mac": "00:90:fb:76:e4:7b",
        "Name": "Ethernet32"
      },
      {
        "Site": "urn:ogf:network:nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_SDSC:edgecore_s0",
        "Peer": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500",
        "Vlan": "3603",
        "Mac": "00:90:fb:76:e4:7b",
        "Name": "PortChannel500"
      },
      {
        "Site": "urn:ogf:network:sc-test.cenic.net:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel501",
        "IPv4": "?port_ipv4?",
        "Node": "NRM_CENIC:aristaeos_s0",
        "Peer": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_102",
        "Vlan": "3603",
        "Mac": "28:e7:1d:3f:53:88",
        "Name": "Port-Channel501"
      },
      {
        "Site": "urn:ogf:network:sc-test.cenic.net:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502",
        "IPv4": "?port_ipv4?",
        "Node": "NRM_CENIC:aristaeos_s0",
        "Peer": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502",
        "Vlan": "3603",
        "Mac": "28:e7:1d:3f:53:70",
        "Name": "Port-Channel502"
      },
      {
        "Site": "urn:ogf:network:tier2.ultralight.org:2024",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech:dellos10_s0",
        "Peer": "urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103",
        "Vlan": "3603",
        "Mac": "8c:04:ba:e9:0e:a8",
        "Name": "Port-channel 101"
      },
      {
        "Site": "urn:ogf:network:tier2.ultralight.org:2024",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_102",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech:dellos10_s0",
        "Peer": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel501",
        "Vlan": "3603",
        "Mac": "8c:04:ba:e9:0e:a9",
        "Name": "Port-channel 102"
      },
      {
        "Site": "urn:ogf:network:ultralight.org:2013",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:ultralight.org:2013:dellos9_s0:hundredGigE_1-32",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech_Test:dellos9_s0",
        "Peer": "?peer?",
        "Host": [
          {
            "IPv6": "?ipv6?",
            "IPv4": "10.251.87.81/30",
            "Interface": "mlx5p1s1",
            "Mac": "ec:0d:9a:c1:ba:60",
            "Name": "T2_US_Caltech_Test:sandie-7.ultralight.org"
          }
        ],
        "Vlan": "3603",
        "Mac": "4c:76:25:e8:44:c2",
        "Name": "hundredGigE 1/32"
      },
      {
        "Site": "urn:ogf:network:ultralight.org:2013",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech_Test:dellos9_s0",
        "Peer": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101",
        "Vlan": "3603",
        "Mac": "4c:76:25:e8:44:c2",
        "Name": "Port-channel 103"
      }
    ]
  }

@sunami09
Copy link
Collaborator 

{
    "Ports": [
      {
        "Site": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_UCSD_OASIS:oasis",
        "Peer": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502",
        "Vlan": "3603",
        "Mac": "?port_mac?",
        "Name": "Pc502"
      },
      {
        "Site": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_UCSD_OASIS:oasis",
        "Peer": "urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500",
        "Vlan": "3603",
        "Mac": "?port_mac?",
        "Name": "Pc500"
      },
      {
        "Site": "urn:ogf:network:nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:Ethernet32",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_SDSC:edgecore_s0",
        "Peer": "?peer?",
        "Host": [
          {
            "IPv6": "?ipv6?",
            "IPv4": "10.251.87.82/30",
            "Interface": "enp168s0np0",
            "Mac": "a0:88:c2:86:ee:7c",
            "Name": "T2_US_SDSC:k8s-gen5-01.sdsc.optiputer.net"
          }
        ],
        "Vlan": "3603",
        "Mac": "00:90:fb:76:e4:7b",
        "Name": "Ethernet32"
      },
      {
        "Site": "urn:ogf:network:nrp-nautilus.io:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_SDSC:edgecore_s0",
        "Peer": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500",
        "Vlan": "3603",
        "Mac": "00:90:fb:76:e4:7b",
        "Name": "PortChannel500"
      },
      {
        "Site": "urn:ogf:network:sc-test.cenic.net:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel501",
        "IPv4": "?port_ipv4?",
        "Node": "NRM_CENIC:aristaeos_s0",
        "Peer": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_102",
        "Vlan": "3603",
        "Mac": "28:e7:1d:3f:53:88",
        "Name": "Port-Channel501"
      },
      {
        "Site": "urn:ogf:network:sc-test.cenic.net:2020",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502",
        "IPv4": "?port_ipv4?",
        "Node": "NRM_CENIC:aristaeos_s0",
        "Peer": "urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502",
        "Vlan": "3603",
        "Mac": "28:e7:1d:3f:53:70",
        "Name": "Port-Channel502"
      },
      {
        "Site": "urn:ogf:network:tier2.ultralight.org:2024",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech:dellos10_s0",
        "Peer": "urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103",
        "Vlan": "3603",
        "Mac": "8c:04:ba:e9:0e:a8",
        "Name": "Port-channel 101"
      },
      {
        "Site": "urn:ogf:network:tier2.ultralight.org:2024",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_102",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech:dellos10_s0",
        "Peer": "urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel501",
        "Vlan": "3603",
        "Mac": "8c:04:ba:e9:0e:a9",
        "Name": "Port-channel 102"
      },
      {
        "Site": "urn:ogf:network:ultralight.org:2013",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:ultralight.org:2013:dellos9_s0:hundredGigE_1-32",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech_Test:dellos9_s0",
        "Peer": "?peer?",
        "Host": [
          {
            "IPv6": "?ipv6?",
            "IPv4": "10.251.87.81/30",
            "Interface": "mlx5p1s1",
            "Mac": "ec:0d:9a:c1:ba:60",
            "Name": "T2_US_Caltech_Test:sandie-7.ultralight.org"
          }
        ],
        "Vlan": "3603",
        "Mac": "4c:76:25:e8:44:c2",
        "Name": "hundredGigE 1/32"
      },
      {
        "Site": "urn:ogf:network:ultralight.org:2013",
        "IPv6": "?port_ipv6?",
        "Port": "urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103",
        "IPv4": "?port_ipv4?",
        "Node": "T2_US_Caltech_Test:dellos9_s0",
        "Peer": "urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101",
        "Vlan": "3603",
        "Mac": "4c:76:25:e8:44:c2",
        "Name": "Port-channel 103"
      }
    ]
  }

sunami09 pushed a commit that referenced this issue Oct 21, 2024
Diagram class related to #127
08dd2c0
@sunami09
Copy link
Collaborator

For Manifest:

{'Ports': [{'Host': [{'IPv4': '10.251.86.138/30',
                      'IPv6': '?ipv6?',
                      'Interface': 'enp168s0np0',
                      'Mac': 'a0:88:c2:86:ee:7c',
                      'Name': 'T2_US_SDSC:k8s-gen5-01.sdsc.optiputer.net'}],
            'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '00:90:fb:76:e4:7b',
            'Name': 'Ethernet32',
            'Node': 'T2_US_SDSC:edgecore_s0',
            'Peer': '?peer?',
            'Port': 'urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:Ethernet32',
            'Site': 'urn:ogf:network:nrp-nautilus.io:2020',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '00:90:fb:76:e4:7b',
            'Name': 'PortChannel500',
            'Node': 'T2_US_SDSC:edgecore_s0',
            'Peer': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500',
            'Port': 'urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500',
            'Site': 'urn:ogf:network:nrp-nautilus.io:2020',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '4c:76:25:e8:44:c2',
            'Name': 'Port-channel 103',
            'Node': 'T2_US_Caltech_Test:dellos9_s0',
            'Peer': 'urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101',
            'Port': 'urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103',
            'Site': 'urn:ogf:network:ultralight.org:2013',
            'Vlan': '3607'},
           {'Host': [{'IPv4': '10.251.86.137/30',
                      'IPv6': '?ipv6?',
                      'Interface': 'mlx5p1s1',
                      'Mac': 'ec:0d:9a:c1:ba:60',
                      'Name': 'T2_US_Caltech_Test:sandie-7.ultralight.org'}],
            'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '4c:76:25:e8:44:c2',
            'Name': 'hundredGigE 1/32',
            'Node': 'T2_US_Caltech_Test:dellos9_s0',
            'Peer': '?peer?',
            'Port': 'urn:ogf:network:ultralight.org:2013:dellos9_s0:hundredGigE_1-32',
            'Site': 'urn:ogf:network:ultralight.org:2013',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '28:e7:1d:3f:53:70',
            'Name': 'Port-Channel502',
            'Node': 'NRM_CENIC:aristaeos_s0',
            'Peer': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502',
            'Port': 'urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502',
            'Site': 'urn:ogf:network:sc-test.cenic.net:2020',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '28:e7:1d:3f:53:88',
            'Name': 'Port-Channel501',
            'Node': 'NRM_CENIC:aristaeos_s0',
            'Peer': 'urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_102',
            'Port': 'urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel501',
            'Site': 'urn:ogf:network:sc-test.cenic.net:2020',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '8c:04:ba:e9:0e:a8',
            'Name': 'Port-channel 101',
            'Node': 'T2_US_Caltech:dellos10_s0',
            'Peer': 'urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103',
            'Port': 'urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101',
            'Site': 'urn:ogf:network:tier2.ultralight.org:2024',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '8c:04:ba:e9:0e:a9',
            'Name': 'Port-channel 102',
            'Node': 'T2_US_Caltech:dellos10_s0',
            'Peer': 'urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel501',
            'Port': 'urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_102',
            'Site': 'urn:ogf:network:tier2.ultralight.org:2024',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '?port_mac?',
            'Name': 'Pc500',
            'Node': 'T2_US_UCSD_OASIS:oasis',
            'Peer': 'urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500',
            'Port': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500',
            'Site': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020',
            'Vlan': '3607'},
           {'IPv4': '?port_ipv4?',
            'IPv6': '?port_ipv6?',
            'Mac': '?port_mac?',
            'Name': 'Pc502',
            'Node': 'T2_US_UCSD_OASIS:oasis',
            'Peer': 'urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502',
            'Port': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502',
            'Site': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020',
            'Vlan': '3607'}]}

image

@sunami09
Copy link
Collaborator

@juztas hey what is this error,

Mon, 28 Oct 2024 03:35:11.132 - RTMonLibs.LogLib - INFO - ================================================================================
Mon, 28 Oct 2024 03:35:11.132 - RTMonLibs.LogLib - INFO - Submit Execution: rtmon-debug-sense-o-dev.es.net-6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee, {'state': 'submitted', 'referenceUUID': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee', 'orchestrator': 'sense-o-dev.es.net', 'submission': 'AUTH_KEY'}
Mon, 28 Oct 2024 03:35:11.875 - RTMonLibs.LogLib - INFO - Here is instance for 6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee:
Mon, 28 Oct 2024 03:35:11.877 - RTMonLibs.LogLib - INFO - {'alias': 'RTMON-Caltech-SDSC-v2',
 'archived': False,
 'intents': [{'creation_time': '2024-10-22 16:12:00',
              'id': '7fdaf618-b3e6-4f5b-864e-acaa7e456362',
              'json': {'data': {'connections': [{'bandwidth': {'capacity': '1000',
                                                               'qos_class': 'guaranteedCapped'},
                                                 'ip_address_pool': {'name': 'AutoGOLE-IPv4-Test-Pool',
                                                                     'netmask': '/30'},
                                                 'name': 'Connection 1',
                                                 'terminals': [{'assign_ip': True,
                                                                'uri': 'urn:ogf:network:ultralight.org:2013:sandie-1.ultralight.org',
                                                                'vlan_tag': 'any'},
                                                               {'assign_ip': True,
                                                                'uri': 'urn:ogf:network:nrp-nautilus.io:2020:k8s-gen5-02.sdsc.optiputer.net',
                                                                'vlan_tag': 'any'}]}],
                                'type': 'Multi-Path P2P VLAN'},
                       'options': [],
                       'queries': [],
                       'service': 'dnc',
                       'service_instance_uuid': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee',
                       'service_profile_uuid': 'ca660eb3-899c-49bf-8e18-d5c0dcbb9fdf'},
              'provisioned': True,
              'serviceDeltaUUID': '5a89c625-fe31-4f34-80d5-6a94c09762be',
              'serviceInstanceUUID': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee'}],
 'lastState': 'COMMITTED',
 'owner': '[email protected]',
 'profileUUID': 'ca660eb3-899c-49bf-8e18-d5c0dcbb9fdf',
 'referenceUUID': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee',
 'state': 'CREATE - READY',
 'timestamp': '2024/10/22 16:12:00'}
Traceback (most recent call last):
  File "src/python/RTMon/worker.py", line 271, in <module>
    worker.startwork()
  File "src/python/RTMon/worker.py", line 230, in startwork
    self._startwork()
  File "src/python/RTMon/worker.py", line 253, in _startwork
    self.main()
  File "src/python/RTMon/worker.py", line 217, in main
    self.submit_exe(filename, fout)
  File "src/python/RTMon/worker.py", line 50, in submit_exe
    manifest = self.s_getManifest(instance)
  File "/opt/devrtmon/src/python/RTMonLibs/SenseAPI.py", line 79, in s_getManifest
    response = wApi.manifest_create(dumpJson(template, self.logger))
  File "/usr/local/lib/python3.6/site-packages/sense/common.py", line 21, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sense/client/workflow_combined_api.py", line 680, in manifest_create
    body_xml, **kwargs)  # noqa: E501
  File "/usr/local/lib/python3.6/site-packages/sense/common.py", line 21, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sense/client/workflow_combined_api.py", line 735, in instance_si_uuid_manifest_post_with_http_info
    query_params=query_params)
  File "/usr/local/lib/python3.6/site-packages/sense/common.py", line 21, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sense/client/requestwrapper.py", line 145, in request
    f"Returned code {ret.status_code} with error '{error_message}'")
ValueError: Returned code 500 with error 'Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (io.undertow.servlet.spec.ServletInputStreamImpl); line: 1, column: 2]'
[root@docker-desktop devrtmon]#

@sunami09
Copy link
Collaborator

@juztas

T2_US_Caltech_Test:sandie-1.ultralight.org', 'vlan': 'vlan.3601'}], 'Vlan': '3601', 'Mac': '4c:76:25:e8:44:c2', 'Name': 'hundredGigE 1/10'}, {'Site': 'urn:ogf:network:ultralight.org:2013', 'IPv6': '?port_ipv6?', 'Port': 'urn:ogf:network:ultralight.org:2013:dellos9_s0:Port-channel_103', 'IPv4': '?port_ipv4?', 'Node': 'T2_US_Caltech_Test:dellos9_s0', 'Peer': 'urn:ogf:network:tier2.ultralight.org:2024:dellos10_s0:Port-channel_101', 'Vlan': '3601', 'Mac': '4c:76:25:e8:44:c2', 'Name': 'Port-channel 103'}, {'Site': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020', 'IPv6': '?port_ipv6?', 'Port': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc502', 'IPv4': '?port_ipv4?', 'Node': 'T2_US_UCSD_OASIS:oasis', 'Peer': 'urn:ogf:network:sc-test.cenic.net:2020:aristaeos_s0:Port-Channel502', 'Vlan': '3601', 'Mac': '?port_mac?', 'Name': 'Pc502'}, {'Site': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020', 'IPv6': '?port_ipv6?', 'Port': 'urn:ogf:network:sense-oasis-nrp-nautilus.io:2020:oasis:Pc500', 'IPv4': '?port_ipv4?', 'Node': 'T2_US_UCSD_OASIS:oasis', 'Peer': 'urn:ogf:network:nrp-nautilus.io:2020:edgecore_s0:PortChannel500', 'Vlan': '3601', 'Mac': '?port_mac?', 'Name': 'Pc500'}]}, 'dashbInfo': {'uid': '593b0eb9-f5bf-868b-9ed3-ec76ca995613', 'annotation_panels': [306, 307, 308, 409, 410, 411, 512, 513, 514, 615, 616, 617, 718, 719, 720, 821, 822, 823, 924, 925, 926, 1034, 1035, 1036, 1037, 1038, 1039, 1040]}, 'retries': 0}
Mon, 28 Oct 2024 03:37:19.528 - RTMonLibs.LogLib - INFO - Deleting Dashboard: RTMON-Caltech-SDSC-v2|Flow: 7fdaf618-b3e6-4f5b-864e-acaa7e456362|2024/10/22 16:12:00 (Sunami-Prod)
Traceback (most recent call last):
  File "src/python/RTMon/worker.py", line 271, in <module>
    worker.startwork()
  File "src/python/RTMon/worker.py", line 230, in startwork
    self._startwork()
  File "src/python/RTMon/worker.py", line 253, in _startwork
    self.main()
  File "src/python/RTMon/worker.py", line 219, in main
    self.delete_exe(filename, fout)
  File "src/python/RTMon/worker.py", line 104, in delete_exe
    self.g_deleteDashboard(dashbName)
TypeError: g_deleteDashboard() missing 1 required positional argument: 'folderTitle'

sense-rtmon/autogole-api/src/python/RTMon/worker.py

Line 98 in 96b33b3

self.g_deleteDashboard(dashbName)

i think it need 2 arguments:

definition:

sense-rtmon/autogole-api/src/python/RTMonLibs/GrafanaAPI.py

Line 87 in 96b33b3

def g_deleteDashboard(self, title, folderTitle):

@juztas
Copy link
Collaborator Author

juztas commented Oct 28, 2024

@juztas hey what is this error,

Mon, 28 Oct 2024 03:35:11.132 - RTMonLibs.LogLib - INFO - ================================================================================
Mon, 28 Oct 2024 03:35:11.132 - RTMonLibs.LogLib - INFO - Submit Execution: rtmon-debug-sense-o-dev.es.net-6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee, {'state': 'submitted', 'referenceUUID': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee', 'orchestrator': 'sense-o-dev.es.net', 'submission': 'AUTH_KEY'}
Mon, 28 Oct 2024 03:35:11.875 - RTMonLibs.LogLib - INFO - Here is instance for 6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee:
Mon, 28 Oct 2024 03:35:11.877 - RTMonLibs.LogLib - INFO - {'alias': 'RTMON-Caltech-SDSC-v2',
 'archived': False,
 'intents': [{'creation_time': '2024-10-22 16:12:00',
              'id': '7fdaf618-b3e6-4f5b-864e-acaa7e456362',
              'json': {'data': {'connections': [{'bandwidth': {'capacity': '1000',
                                                               'qos_class': 'guaranteedCapped'},
                                                 'ip_address_pool': {'name': 'AutoGOLE-IPv4-Test-Pool',
                                                                     'netmask': '/30'},
                                                 'name': 'Connection 1',
                                                 'terminals': [{'assign_ip': True,
                                                                'uri': 'urn:ogf:network:ultralight.org:2013:sandie-1.ultralight.org',
                                                                'vlan_tag': 'any'},
                                                               {'assign_ip': True,
                                                                'uri': 'urn:ogf:network:nrp-nautilus.io:2020:k8s-gen5-02.sdsc.optiputer.net',
                                                                'vlan_tag': 'any'}]}],
                                'type': 'Multi-Path P2P VLAN'},
                       'options': [],
                       'queries': [],
                       'service': 'dnc',
                       'service_instance_uuid': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee',
                       'service_profile_uuid': 'ca660eb3-899c-49bf-8e18-d5c0dcbb9fdf'},
              'provisioned': True,
              'serviceDeltaUUID': '5a89c625-fe31-4f34-80d5-6a94c09762be',
              'serviceInstanceUUID': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee'}],
 'lastState': 'COMMITTED',
 'owner': '[email protected]',
 'profileUUID': 'ca660eb3-899c-49bf-8e18-d5c0dcbb9fdf',
 'referenceUUID': '6803cb62-8ec0-4b70-a6f0-26a1fd2d19ee',
 'state': 'CREATE - READY',
 'timestamp': '2024/10/22 16:12:00'}
Traceback (most recent call last):
  File "src/python/RTMon/worker.py", line 271, in <module>
    worker.startwork()
  File "src/python/RTMon/worker.py", line 230, in startwork
    self._startwork()
  File "src/python/RTMon/worker.py", line 253, in _startwork
    self.main()
  File "src/python/RTMon/worker.py", line 217, in main
    self.submit_exe(filename, fout)
  File "src/python/RTMon/worker.py", line 50, in submit_exe
    manifest = self.s_getManifest(instance)
  File "/opt/devrtmon/src/python/RTMonLibs/SenseAPI.py", line 79, in s_getManifest
    response = wApi.manifest_create(dumpJson(template, self.logger))
  File "/usr/local/lib/python3.6/site-packages/sense/common.py", line 21, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sense/client/workflow_combined_api.py", line 680, in manifest_create
    body_xml, **kwargs)  # noqa: E501
  File "/usr/local/lib/python3.6/site-packages/sense/common.py", line 21, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sense/client/workflow_combined_api.py", line 735, in instance_si_uuid_manifest_post_with_http_info
    query_params=query_params)
  File "/usr/local/lib/python3.6/site-packages/sense/common.py", line 21, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sense/client/requestwrapper.py", line 145, in request
    f"Returned code {ret.status_code} with error '{error_message}'")
ValueError: Returned code 500 with error 'Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (io.undertow.servlet.spec.ServletInputStreamImpl); line: 1, column: 2]'
[root@docker-desktop devrtmon]#

New issue created here: sdn-sense/sense-o-py-client#35

@juztas juztas mentioned this issue Oct 28, 2024
Closed
@xi-yang xi-yang mentioned this issue Oct 28, 2024
Closed
@juztas juztas added this to the RTMon v0.2 milestone Oct 28, 2024
@sunami09
Copy link
Collaborator

sunami09 commented Nov 1, 2024 

?port_mac?', 'Name': 'Pc502'}]}, 'dashbInfo': {'uid': '9019f789-2c56-254d-bf84-9734b2eac740', 'annotation_panels': [306, 307, 308, 409, 410, 411, 512, 513, 514, 615, 616, 617, 718, 719, 720, 821, 822, 823, 924, 925, 926, 1034, 1035, 1036, 1037, 1038, 1039, 1040]}, 'retries': 0}
Fri, 01 Nov 2024 17:27:32.633 - RTMonLibs.LogLib - INFO - Removed diagram image /srv/images/diagram_0a303274-a293-4af8-88b7-5b010d23a0f7.png
Fri, 01 Nov 2024 17:27:32.634 - RTMonLibs.LogLib - INFO - Deleting Dashboard: RTMON-IT6-Caltech-SDSC|Flow: 5d86903e-6b09-4c61-8cca-4d7e36c9aa1a|2024/10/22 16:17:47 (Sunami-Prod)
Fri, 01 Nov 2024 17:27:32.634 - RTMonLibs.LogLib - INFO - Deleting Dashboard: RTMON-IT6-Caltech-SDSC|Flow: 5d86903e-6b09-4c61-8cca-4d7e36c9aa1a|2024/10/22 16:17:47
Fri, 01 Nov 2024 17:27:32.683 - RTMonLibs.LogLib - ERROR - Failed to delete dashboard RTMON-IT6-Caltech-SDSC|Flow: 5d86903e-6b09-4c61-8cca-4d7e36c9aa1a|2024/10/22 16:17:47: Client Error 404: Dashboard not found
Fri, 01 Nov 2024 17:27:33.735 - RTMonLibs.LogLib - ERROR - Failed to delete dashboard RTMON-IT6-Caltech-SDSC|Flow: 5d86903e-6b09-4c61-8cca-4d7e36c9aa1a|2024/10/22 16:17:47: Client Error 404: Dashboard not found
Fri, 01 Nov 2024 17:27:34.791 - RTMonLibs.LogLib - ERROR - Failed to delete dashboard RTMON-IT6-Caltech-SDSC|Flow: 5d86903e-6b09-4c61-8cca-4d7e36c9aa1a|2024/10/22 16:17:47: Client Error 404: Dashboard not found
Traceback (most recent call last):
  File "src/python/RTMon/worker.py", line 272, in <module>
    worker.startwork()
  File "src/python/RTMon/worker.py", line 231, in startwork
    self._startwork()
  File "src/python/RTMon/worker.py", line 254, in _startwork
    self.main()
  File "src/python/RTMon/worker.py", line 220, in main
    self.delete_exe(filename, fout)
  File "src/python/RTMon/worker.py", line 105, in delete_exe
    self.g_deleteDashboard(dashbName, folderName)
  File "/opt/devrtmon/src/python/RTMonLibs/GrafanaAPI.py", line 100, in g_deleteDashboard
    raise Exception(f"Failed to delete dashboard {title} after 3 retries")
Exception: Failed to delete dashboard RTMON-IT6-Caltech-SDSC|Flow: 5d86903e-6b09-4c61-8cca-4d7e36c9aa1a|2024/10/22 16:17:47 after 3 retries

when I removed acces from sense-rtmon

@juztas
Copy link
Collaborator Author

juztas commented Nov 1, 2024

It is just a warning - and once deleted it will not complain. Given the fact that there might be multiple rtmon's running (and also you might have supervisor service running) - supervisor deleted it already - but your script started later and tried to do same. Important is to check if dashboard is deleted in grafana (what I can see it is)

juztas pushed a commit that referenced this issue Nov 8, 2024
@juztas
Diagram class related to #127
29eeecb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sunami09 sunami09

Labels
v0.2
Projects
None yet
Milestone
RTMon v0.2
Development

No branches or pull requests
2 participants
@juztas @sunami09