Web crawler API for k6

xk6-crawler v0.1.0 (the first public release) is here !

Web crawler API for k6

A programmable web crawler, which enables testing of web sites in Grafana k6 using the visitor pattern. The JavaScript API allows you to create and easily program web crawlers. In the callback functions of the crawler, custom logic can be used for testing.

import { Crawler } from "k6/x/crawler";

export default function () {
  const c = new Crawler({ max_depth: 2 });

  c.onHTML("a[href]", (e) => {
    if (e.attr("href").startsWith("/")) {
      e.request.visit(e.attr("href"));
    }
  });

  c.onResponse((r) => {
    console.log(r.status_code, r.request.url);
  });

  c.visit("https://grafana.com");
}```

https://github.com/grafana/xk6-crawler

1 post - 1 participant

Read full topic