Add support for socket.io

[peixin]

Server using Socket.IO

Socket.IO is not a WebSocket implementation. Although Socket.IO indeed uses WebSocket as a transport when possible, it adds some metadata to each packet: the packet type, the namespace and the ack id when a message acknowledgement is needed. That is why a WebSocket client will not be able to successfully connect to a Socket.IO server, and a Socket.IO client will not be able to connect to a WebSocket server (like ws://echo.websocket.org) either. Please see the protocol specification here.

Useing k6 ws module connect will throw an error:

ERRO[0000] GoError: websocket: bad handshake

[na--]

Thanks for posting this issue! Socket.io support has been requested a few times, or rather people have asked questions about it. My previous response has been that we don't support it yet, but maybe writing some wrapper around out websocket support would do the trick - I just now realize how wrong I was ... 😞 I didn't understand that it was completely impossible for a normal websocket client to connect to a socket.io server... 🤦‍♂️

So, we'd like to have socket.io support in k6, eventually... Unfortunately it's probably not going to be very high in our roadmap for the next several months 😞 . One of the reasons is that, in order to support such an asynchronous protocol properly (and to actually improve our current WebSocket support and pave the way for things like gRPC), we need to fully implement global per-VU event loops (#882).

I also did a cursory search for a Go socket.io library and came up with https://github.com/googollee/go-socket.io . It claims to only support version 1.4 of the protocol, whereas the main site advertises 2.0...

[peixin]

Thanks for your reply. I temporary use other tools to test socket.io recently, it doesn't feel easy to use.
Hope to use k6's socket.io feature as soon as possible, then I will migrate the code to k6.

[lemon-li]

@na--
Hello
My server use Node.js socket.io, is there any way to test it?
Is it possible to use nodejs socket.io node_modules through webpack? Or there is no solution ? 😞

[imiric]

@lemon-li It seems that the socket.io client library could be bundled with Webpack (see here, assuming that still works), so you might be able to import it in k6, but as @na-- mentioned, you'll probably run into issues because the k6 JS runtime lacks a global event loop, which is needed for these async protocols.

Once that functionality is in k6, this might be a good candidate to implement as a plugin. So several things need to align before this is officially supported.

[romanlv]

using pure WebSocket implementation to connect to socket.io server worked for me, the key here is to inspect in browser dev tools to see what is the actual url used to connect and what are the messages sent between client and server. You script should simulate this

e.g.

import ws from "k6/ws";
import { check } from "k6";


export default function () {
  const { CHAT_WS } = __ENV;
  const token =
    "....token...";
  const url = `${CHAT_WS}/socket.io/?token=${token}&tokenType=jwt&EIO=3&transport=websocket`;

  var response = ws.connect(url, {}, function (socket) {
    socket.on("open", function open() {
      // console.log("connected");
      // socket.send(Date.now());

      socket.setInterval(function timeout() {
        socket.ping();
        console.log("Pinging every 5sec (setInterval test)");
      }, 1000 * 5);
    });

    socket.on("message", function incoming(msg) {
      // console.log(msg);
      if (msg === "40") {
        socket.send("40/chat?token=" + token + "&tokenType=jwt,");
      }

      if (msg === "40/chat") {
        socket.send(
          '42/chat,["ACTION_1",{"data": "x"}]'
        );
      }

      if (msg && msg.startsWith("42/chat,")) {
        const data = JSON.parse(msg.substring("42/chat,".length));
        const action = data[0];
        console.log(`received msg: ${action}`);
    
        // waiting for specific action and responding to it
        if (action === "ACTION_X") {
          const chatMsg = `k6 hello, vu=${__VU}, iter=${__ITER}, ${Date.now().toString()}`;
          socket.send(
            `42/chat,["MESSAGE_ACTION",{"message": "${chatMsg}"}]`
          );
        }
      }
    });

    socket.on("close", function close() {
      console.log("disconnected");
    });

    socket.on("error", function (e) {
      console.log("error", e);
      if (e.error() != "websocket: close sent") {
        console.log("An unexpected error occured: ", e.error());
      }
    });

    socket.setTimeout(function () {
      console.log("60 seconds passed, closing the socket");
      socket.close();
    }, 1000 * 60);
  });

  check(response, { "status is 101": (r) => r && r.status === 101 });
}

/chat is the custom namespace used in my case, it is empty for default namespace.
I have no idea what those 40 and 42 codes mean, it's just something socket.io adds internally

[wreulicke]

I wrote a minimal packet parser for socket.io/engine.io.
It does not support edge cases.

https://gist.github.com/wreulicke/e05b42ba79f42768a54f3b2a9cb7c416

And also, I tried to use socket.io-parser and engine.io-parser with some bundle tools for webpack/parcel/etc.
But, does not work for me...
(I don't try to use it with browserify. please let me know if working)

[qwertynik]

@na-- Any plans for support being added for socket.io?

[imiric]

@qwertynik There are no concrete plans for this, but have you tried the workarounds mentioned above?

If those don't work for you, you can try creating an xk6 extension for it, though you might run into issues with supporting the async workflows. See this article for an introduction.

[qwertynik]

@imiric
Have not tried the workarounds yet. Was looking for testing tools to test socket.io. Landed upon a couple of tools. Using jMeter for now.
Thanks for the link to extension development. It is highly unlikely that I will be building one at least in the near future.

[hartatovich]

if anyone still looking for a solution i would love to collaborate together and work something out

i would love to have socket.io support in k6

[mstoykov]

just FYI there is an old PR (with an unknown working state) that can probably be made into an extension.

[ppcano]

Here is another example and JS utils to support k6/ws to connect to a socket.io application. Kudos to @safebear

[Stefanieissun]

it's not okay, when I use k6/ws
not connected in server

[vvargas90]

I have the following error after exactly 30 seconds: websocket: close 1005 (no status), but for the rest the implementation works fine!

[codebien]

Thanks to @andrew-delph and @sennett-lau there are now some solutions in grafana/xk6-websockets#34 built on top of WebSocket module. I encourage you to try them and report any eventual problem in dedicated issues.

For now, we don't plan to implement socket.io directly in k6-core, so I'm closing the issue.

[CSenshi]

Neither of the solutions work with custom parsers for socket io which is widely used. We integrated this solution in our project but after implementing custom parsers in our projects this perf test scripts are having no value

[mstoykov]

Hi @CSenshi, this issue is closed not so much because there are solution, but because we do not intend on working on it.

Socket.io isn't a standard, and while fairly popular, not standards are very unlikely to enter into k6 core. Having said that I expect at some future point the default socket io implementation to just wokr better as we keep implementing standards. That is unlikely to be soon ™ though.

I have not seen any extension for this either so maybe it isn't as popular as expected 🤷 . The two workaround are not made by anyone on the core team, and I don't think any of us use socket io for anything. It will probably be better to try to contact the people who have worked on them, I doubt they are look at this closed issue.

Another option is to write in the community forum in hopes other people might help with this.

[ValentinGuevara]

Hey everyone, just coded a minimal socket.io wrapper on k6 for testing purpose. Here is an example with comments below. I used the same principles than official docs by upgrading connection from polling to websocket and respecting their protocol handshake. You will normally trigger same events than sock-io.client. Just started a socket.io server on my machine with default namespace.

import ws from "k6/ws";
import http from "k6/http";
import { sleep } from "k6";

const socketIoPath = "localhost:3000/socket.io/?EIO=4&transport=";

const formatMessage = (...args) => {
  // https://socket.io/docs/v4/socket-io-protocol/#packet-encoding
  return `42${JSON.stringify(args)}`;
}

export default function () {
  // HTTP then Websocket upgrading
  const received = http.get(`http://${socketIoPath}polling`);
  // substring 1 to remove first char protocol number
  const { sid, upgrades, pingInterval, pingTimeout, maxPayload } = JSON.parse(received.body.substring(1));
  // Server informs websocket available
  if(upgrades.includes("websocket")) {
    // Use sid to upgrade current polling session
    ws.connect(`ws://${socketIoPath}websocket&sid=${sid}`, (socket) => {
      socket.on("open", () => {
        socket.send('2probe'); // Send ping
      });
      socket.on("message", (message) => {
        console.log(message);

        // Receive pong from server
        if(message === "3probe") {
          socket.send('5'); // Upgrade
          socket.send('40'); // Ask default namespace / connection
        }
        // Namespace connected by server
        if(message.startsWith("40")) {
          // Test sending JSON formatted message on event test
          const messageSent = formatMessage("test", JSON.stringify({
            bool: true,
            "test": "success"
          }));
          console.log(`Message sent : ${messageSent}`)
          socket.send(messageSent); // Send message event
          sleep(5);
          socket.send('41'); // Disconnect namespace
          sleep(1);
          socket.close();
        }
      });
      socket.on("error", (error) => {
        console.error(error)
      })
    });
  }
}