dagger.io integration

[caniko]

Background

dagger is very useful for simplifying complex CI/CD pipelines, and sometimes you want to run a few components from these locally during development. This is where mirrord can be very useful.

Feature

mirrord dagger -- <dagger command> that replaces dagger call <dagger command> for cases when we want to bind our dagger call to a remote cluster using mirrord.

Backend

In dagger we define our own CLI commands using the dagger SDKs, and dagger call is used to run these commands from the CL. These commands may spin up several containers using docker, podman, or whatever container runtime with docker interface. This new feature should detect these requests made by dagger to the docker api, and apply mirrord container to them.

[linear]

MBE-1493

[aviramha]

Thanks for sharing! We're thinking about the solution and checking with Dagger's team maybe we can colab.

[marcosnils]

hey @aviramha! we currently have a thread in our Discord here: https://discord.com/channels/707636530424053791/1428375419387641907. Exchanging some ideas with @caniko, I'm unsure what would need to be done from our end here.

Dagger can already run containerized sandboxed processes, so AFAIK you can aready run mirrord within Dagger to proxy whatever you need to your kube environments.

I'd like to understand better @caniko's use-case to see if there's effectively something missing here.

[caniko]

I am afraid the perspectives are mixed up @marcosnils. mirrord doesn’t automatically “capture” or attach to all containers spun up by Dagger from a single call.

I don't want to run mirrord inside dagger, but dagger inside mirrord instead.

[marcosnils]

I am afraid the perspectives are mixed up @marcosnils. mirrord doesn’t automatically “capture” or attach to all containers spun up by Dagger from a single call.

this is what I'm trying to understand in our Discord thread. Does it make sense for mirrord to proxy *ALL* containers that Dagger launches? Dagger does a bunch of stuff under the hood compared to other container runtimes like Docker. That's why my initial approach to this problem was to run mirrord within Dagger so you have better control and scope about the things that you're mirroring.

[caniko]

Does it make sense for mirrord to proxy ALL containers that Dagger launches?

That is a very good question, and I think it is worth exploring before moving further.

Moreover, I would say that there is credence to both of the approaches. If we have to pick one over the other for UX reasons, the SDK approach would be the correct one for reasons outlined above.

[marcosnils]

That is a very good question, and I think it is worth exploring before moving further.

Exactly, that's my main reason for asking a concrete example here because it really depends on what you're trying to achieve. Is there any way you could put a very simple example that summarizes your use-case and how you're intending to use mirrord in combination with Dagger to achieve what you need?

If we have to pick one over the other for UX reasons, the SDK approach would be the correct one for reasons outlined
above.

This makes sense.

[caniko]

Is there any way you could put a very simple example that summarizes your use-case and how you're intending to use mirrord in combination with Dagger to achieve what you need?

Uses of mirrord are never simple because we are interacting with a network outside of our LAN, but I understand the point, will try.

We have production databases that we’d like to run smoke and end-to-end tests against locally while developing those tests, or while developing features related to them. We don’t currently need SDK-level control over how Mirrord is applied to our Dagger function, since our tests are fairly simple for now; moreover, having that control in the future would definitely be valuable.

[marcosnils]

We have production databases that we’d like to run smoke and end-to-end tests against locally while developing those tests, or while developing features related to them.

so IIUC what you need is mostly a way to run your test framework (go test, npm test, etc) which runs in Dagger with mirrord so the database calls that the tests do, get routed to your production databases, correct?