inithooks init-fence not functioning

[JedMeister]

The TurnKey init-fence (part of inithooks) is not working.

Part of the issue is that the default config file uses bash arrays but the EnvironmentFile in the service is loaded as a string. That has been addressed in turnkeylinux/inithooks#70.

The other part of the issue is that simplehttpd.py (minimalist web server which serves the page displayed when the fence is enabled) is being called. I had a quick look at that, but simplehttpd.py doesn't work as I expected.

Further consideration makes me inclined to refactor the whole thing and combine all functionality into a single script (i.e. the init-fence executable and the simplehttpd.py script). AFAIK we aren't using simplehttpd.py anywhere else so that probably makes sense?

[marcos-mendez]

LOL no it really doesn't make sense

I may be off base here, so please feel free to ignore this if my ignorance is greater than my understanding. But I wonder whether this issue might be a good opportunity to rethink the broader approach a bit more radically.

My feeling is that this short init-fence window - where we briefly expose a temporary page before handing things over to the actual application - may no longer be the best model.

Over the last year or so, I’ve been experimenting quite a lot with deploying TurnKey Linux LXC instances directly on public IPv6 - probably on the order of a few hundred instances by now - and that has changed my perspective quite a bit.

With IPv6, we are back to something much closer to the old Internet model: everyone can talk to everyone directly. The problem is that not everyone is really prepared for that. Once init-fence is done and the appliance completes initialization, it is very easy for services such as SQL frontends, Adminer, Postgres, or other web/admin endpoints to end up directly exposed to the public Internet.

And we should never underestimate the user’s ability to choose passwords like admin/admin or 123456. :)

More broadly, I think our assumptions about what “normal Internet exposure” looks like have changed a lot over the past few years. Some of the extra machinery that made sense in the NAT/DNAT/port-forwarding era may be less helpful now, and may even be adding technical debt at exactly the point where we are trying to simplify for v19.

So I wonder whether the simpler long-term model might be something like this:

• before initialization is complete, expose nothing publicly at all (except perhaps optional SSH);
• after initialization, expose only the appliance's primary user-facing service;
• keep admin surfaces and database-related tools/services (e.g. Webmin, Adminer, phpMyAdmin, direct DB listeners, etc.) disabled, LAN-only, or otherwise non-public by default;
• and then let the user explicitly enable/publicly expose them later via confconsole, in the same general spirit as certificates, SMTP relay, and other post-install configuration.

To me, that feels simpler, safer, and potentially more aligned with the IPv6 reality many ISPs are now creating - at least from what I’ve been seeing in Brazil, where things can suddenly become globally reachable without people really realizing it.

I realize this touches project philosophy, so I may be overreaching a bit here. But I thought it might be worth raising: perhaps instead of spending too much effort preserving the old fence model, we could consider whether removing that layer altogether - and adopting a more explicit "secure by default, expose by choice" model - would both reduce technical debt for v19 and improve the default security posture.

Just a thought.

[OnGle]

I agree, the main issue here is how we communicate this with the user. For interactive installs we can simply provide a dialog at the end of inithooks, or a message in the confconsole opening screen that makes this clear, but for non-interactive installs it might just seem like the appliance is "broken" to the user. This is something we can handle for hub users once we have the new hub, but for proxmox users, or other manual pre-seeding setups it might not be so easy.

[JedMeister]

Thanks guys.

FWIW I have thought for a long time that there should at least be a way to easily disable Webmin and/or Adminer. So the MVP is to have a script that does the work - i.e. toggle service/s and add/remove FW rules.

With a cli tool/s that can actually make the appropriate changes, adding a confconsole module (for easy user toggling) and an interactive inithook (for firstboot config) would be trivial. Adding support for inithook preseeding could be added at the same time.

Although as @OnGle noted, there are "complications" about disabling by default - so at least initially we'd need to leave the default as "enabled" (if it isn't explicitly disabled interactively or via pre-seed).

But IMO at least making it easy to disable - and ideally asking users who run the firstboot scripts interactively is a great first step.

[l-arnold]

From the user perspective, it seems important to enable/disable or expose public/keep private, those controls into an accessible place and ConfConsole, assuming it can reliably be accessed, seems the starting place for this. It may be the "only place" but technically there needs to be an absolutely clear and reliable way to put keys in place so the shell can be accessed no matter.

@marcos-mendez your comment about IP6 exposing everything is interesting. Is that actually the case however. Do most Routers simply open up IP6? That seems possibly the case in that I have seen IP6 traverse systems where IP4 has not.

As to very simple passwords, there are some controls in the TKL system around that, but I will note that I have found ways to get around those controls with subsequent turnkey-init commands.

[marcos-mendez]

From the user perspective, it seems important to enable/disable or expose public/keep private, those controls into an accessible place and ConfConsole, assuming it can reliably be accessed, seems the starting place for this. It may be the "only place" but technically there needs to be an absolutely clear and reliable way to put keys in place so the shell can be accessed no matter.

This is exactly what I tried to address here: #2089, implemented in this commit: 24a64be

The implementation includes a firstboot inithook that detects public IPv6 and prompts the user to restrict access, plus a Confconsole plugin to toggle the firewall afterwards. This works well for a single appliance, but it should become a standard feature in turnkeylinux-apps/core — this involves @JedMeister and @OnGle to make it part of every appliance.
This will impact many applications as IPv6 adoption grows.

@marcos-mendez your comment about IP6 exposing everything is interesting. Is that actually the case however. Do most Routers simply open up IP6? That seems possibly the case in that I have seen IP6 traverse systems where IP4 has not.

Yes. The entire design of IPv6 is to eliminate NAT. Every device gets a globally routable public address — that's the point. You can configure firewall rules at the gateway or use NAT66, but that defeats the purpose of IPv6. It's the network equivalent of using a match to light a kerosene lamp when everyone already has LED bulbs.

In practice, this means any TKL appliance deployed on a network with IPv6 (SLAAC or DHCPv6) is directly reachable from the internet — no NAT protection by default. When a specific application needs to be isolated, it gets assigned a ULA (private) IPv6 prefix, making it unreachable from outside. This is the mindset sysadmins need to adopt.

As to very simple passwords, there are some controls in the TKL system around that, but I will note that I have found ways to get around those controls with subsequent turnkey-init commands.

I added a generate_password() feature to address this: inithooks#71
It uses Python's secrets module (CSPRNG) and guarantees complexity. A review would be very welcome!