Skip to content
← All articles

Systems administration

Protect access to servers and critical infrastructure with €0 in licence fees

12 min read

At Change Healthcare, stolen credentials opened a remote portal with no second factor. Nine days later, the attackers deployed ransomware. Where should you put controls so that a stolen password alone cannot open access to servers? [1]

In 30 seconds

  • Technicians and contractors connect through Guacamole with a second factor. The firewall blocks direct access from the internet and the office LAN.
  • The systems administrator uses a dedicated physical workstation with no internet access. It reaches approved servers on its own VLAN.
  • Guacamole costs €0 in licence fees. You still need to budget for hosting, maintenance and keeping recordings. [2]
A man in a suit, seen from behind, holds a tablet in a data centre.
AI-generated illustration.

Contractors use Guacamole, and administrators use a dedicated workstation

At work, technicians and contractors needed to service servers. I required them to use Guacamole with a second factor on the Debian portal. The firewall blocked their other routes. For systems administration, I reserved a physical workstation on another network.

For technical work, the technician or contractor opens the portal in a browser, authenticates with the TOTP second factor and selects a connection they are authorised to use. Guacamole opens the session to the target. This required route lets us associate the work with an account and review what happened.

For systems administration, the dedicated workstation connects directly to servers through a reserved VLAN isolated from the office network. It has no inbound or outbound internet access but can still reach the systems it manages. A Guacamole outage does not cut off this administration path.

ANSSI recommendations R9 and R10 cover dedicated admin workstations and blocking inbound and outbound internet access. The bastion in the diagram below is the Guacamole jump server. [3]

Two permitted routes and the direct connections the firewall blocks

Who can administer the servers and infrastructure?
The contractor opens the portal from an approved connection. This article does not detail where that network entry point sits.

The portal checks TOTP and logs connections. The proxy guacd records sessions. On the other route, the physical workstation has no WAN access and reaches targets on its dedicated VLAN. The firewall filters both routes and blocks direct attempts from the internet or the office LAN.

The targets and destination ports are common examples. [4]

RDP example. Direct access to the Windows server on TCP/3389 is blocked from the office LAN and the internet. The technician opens the session through guacd if they have permission. The systems administrator can reach the target directly from the physical workstation on the dedicated VLAN if that service is allowed there. If RDP also uses UDP/3389 or listens on a different port, the firewall rules must cover that traffic. [5]

The portal records work that goes through it

I enabled connection logging and Guacamole session recording. When reconstructing what happened, first identify which route was used.

Where to find a record of the work
RouteRecords to checkWhat they show
Guacamole portalConnection history. [6]Account, selected connection and start and end times.
Session through GuacamoleScreen recording from the proxy, if capture is configured. [7]What was on the screen. Background tasks also require the server logs.
Dedicated admin workstationWorkstation and target server logs, supplemented by firewall logs if enabled.Direct connections through the dedicated VLAN. Guacamole history and recordings do not include them.
Before recording: who can replay the video, and for how long?

A video linked to a contractor’s individual account contains personal data. It may also show other people’s files. Before recording, specify the purpose, establish the lawful basis and check whether connection logs alone would suffice. Limit recording to work where it is justified. [8]

Before the first session, tell participants the purpose of recording, the lawful basis, recipients, retention period and how to exercise their rights. Choose a retention period justified by the purpose, schedule deletion and verify it works. Allow only authorised people to replay recordings and log each viewing. [9]

For technical readers · Where to filter proxy traffic guacd ?

The browser connects to the HTTPS front end. The web application communicates with guacd, the process that opens RDP, SSH or VNC connections to approved targets. Restrict those outbound connections at the firewall to the required addresses and ports.

Only the web application should be able to reach the port used by guacd, which does not authenticate its clients. If the connection crosses a network, enable TLS between the two components. [10]

Require the second factor at portal sign-in

On the Debian server, I enabled Guacamole’s TOTP extension. After signing in for the first time, each user enrols their authenticator app. On later sign-ins they enter the temporary code as well as their first factor. RDP or SSH sessions remain subject to the target’s own permissions.

Secure initial enrolment and phone replacement. Before the factor is bound to the account, anyone who knows the password could enrol their own device. Verify the technician’s or contractor’s identity through an established channel before initial activation or reset. For a lost phone, remove the old secret after verifying identity, then help enrol the new device. Do not exempt contractor accounts or their groups from TOTP. [11]

For technical readers · Which components are needed for TOTP?

The guacamole-auth-totp extension supplements the main authentication method in the web app. Enrolment requires storage. Apache’s database authentication extensions provide it in version 1.6.0.

For a native installation, select the extension for the version deployed, put its JAR file in GUACAMOLE_HOME/extensions/ and schedule an application restart. With the official Docker image, the corresponding setting is TOTP_ENABLED=true. [11]

An alternative that resists code relays: Entra and FIDO2

A fake portal can ask for a TOTP code and relay it to the real portal before it expires. An organisation using Microsoft Entra ID can instead delegate portal authentication to Entra and require phishing-resistant authentication for that application, such as a FIDO2 key. The setup described here used TOTP.

With SAML, the Guacamole extension redirects the user to Entra as the identity provider. Once identity is verified, Guacamole applies connection permissions. With a pilot group, verify that the key opens the portal, a code alone is rejected and no other routine sign-in route bypasses the rule. [12] · [13] · [14].

This alternative requires Microsoft Entra ID P1 or a plan that includes it for users covered by Conditional Access. Any licensing cost comes on top of operating the bastion. [15]

For technical readers · Check OpenID Connect compatibility

Apache also offers an OpenID Connect extension. In Guacamole 1.6.0 it uses the implicit flow. Check that the provider and its policy allow this before choosing it. A suitable extension, such as database authentication, must still supply the connections and their permissions. [16]

See how a FIDO2 key stops a fake portal and how to test the Entra policy.

The decisive test: a valid account must still be blocked from the ordinary LAN

Test the same server from three origins within an authorised scope. The portal should open the technician’s intended session. The physical workstation should reach the server over its dedicated VLAN. The firewall should block direct access from the office LAN or public WAN.

The proxy must also verify the server’s identity

The firewall may permit the correct address while a substitute machine answers there. Before sending credentials, the proxy must verify that the target is the expected server.

For technical readers · Verify the SSH key or RDP certificate

For SSH, get the public host key over a trusted channel, such as the admin console. In Guacamole 1.6.0, declare it in ssh_known_hosts or in the connection parameter host-key . Without one of these controls, Guacamole does not verify the SSH server’s identity.

For RDP, use a certificate with the server’s name from an authority the proxy trusts. Keep verification enabled: ignore-cert=true would accept even an invalid certificate. Guacamole also supports fingerprints approved in advance with cert-fingerprints. [4]

Test a pilot target. First check that an authorised connection works. Then present an unexpected SSH key or untrusted RDP certificate. The proxy should refuse to open the session. For a planned rotation, verify the new key or certificate before updating the configuration.

Test revocation on a test target. Remove a pilot account’s access, then verify it can no longer open a connection. In “Active Sessions”, an authorised administrator can select the active connection and choose “Kill Sessions”. That ends the Guacamole connection. Check the session on the server separately and close it if needed. Work already started may continue, so use target logs alongside the video. [17]

How do you work if Guacamole or Entra goes down?

Give authorised administrators an emergency account local to the target or management domain that does not depend on the portal or Entra. Test it from the dedicated admin workstation on its VLAN. The firewall should still block direct contractor connections during the outage. [18]

Authorised admins must be able to open the emergency vault even when Entra and Guacamole are down. Test the full procedure under those conditions and raise an alert each time the account is used. Afterwards, review the logs and rotate any secret that was disclosed.

Three real attacks and the access routes they used

These investigations show where three controls belong: the firewall in front of servers, the second factor at the portal and traffic filtering between workstations and servers.

FAUST · An RDP server exposed to the internet

In October 2023, Truesec investigated a retailer with a server whose network interface bypassed the firewall. Its local firewall was disabled. An attacker opened an RDP session from the internet using an unused domain admin account. Encryption followed in under 72 hours. [19]

Where to block access? Block direct internet RDP on every server interface, including at the local firewall. From an external network authorised for the test, verify that the server remains unreachable over RDP while the intended Guacamole connection works.

Change Healthcare · A remote portal without MFA

On 12 February 2024, stolen credentials opened a Citrix portal without MFA. According to testimony from UnitedHealth Group’s chief executive, the attackers moved through the systems and exfiltrated data. They deployed ransomware nine days after initial access. [1]

Where to block access? On the Guacamole portal described here, an enrolled account must also provide its TOTP code. A stolen password alone is insufficient. To verify, try the correct password with an incorrect code and check that the portal refuses the session.

Nokoyawa · From a compromised workstation to the domain controller

In November 2022, a fake document infected a workstation with IcedID malware. From that workstation, the attackers used a privileged account to connect to a domain controller over RDP. They then reached backup and file servers over RDP before deploying Nokoyawa ransomware. [20]

Where to stop the pivot? If the firewall blocks RDP from the office LAN and allows it to those targets only from the Guacamole proxy or the physical workstation on the admin VLAN, the connection from the infected workstation fails even with a privileged account.

The fake document could still have infected the first workstation. Other administration routes, including SMB and remote execution tools, also need filtering. Logs and recordings help reconstruct what happened afterwards. They do not close an open port.

The bastion needs patches too

Guacamole is a client of the remote servers. A compromised target can therefore attack the proxy when it connects. Check Point demonstrated this on an older RDP client. Apache advisories also describe later issues affecting VNC and the SSH terminal. [21] · [22].

For technical readers · Match Apache advisories to installed components

RDP vulnerability CVE-2020-9498 was fixed in 1.2.0. Apache later fixed a VNC image handling error in 1.5.4 (CVE-2023-43826) and a terminal sequence validation flaw in 1.6.0 (CVE-2024-35164). In the latter case, a crafted sequence sent over a text connection could execute code with the privileges of guacd.

Inventory versions of the web application, proxy and extensions. Compare them with Apache advisories and check patches for Debian and its libraries. After updating, verify MFA, denied access and replay of a session.

How do you update the dedicated workstation without giving it internet access?

ANSSI recommendation R43 calls for dedicated repository mirrors for the administration environment. A relay in an intermediate zone fetches updates from approved official sources. The workstation only contacts the internal relay over the firewall flow designated for updates. [3]

For example, on a Debian workstation, APT verifies the repository metadata signature and then package hashes. Keep those checks on the internal mirror and test the patch on a pilot workstation before rollout. [23]

AI can help attackers exploit exposed interfaces at scale

Between January and February 2026, Amazon Threat Intelligence observed the compromise of more than 600 FortiGate devices. Attackers took advantage of internet-facing admin interfaces, weak credentials and the lack of a second factor.

AI helped write scripts that sorted stolen configurations and prepared reconnaissance of internal networks. The investigation concerned FortiGate. Amazon did not observe exploitation of a vulnerability in those devices during this campaign. [24]

What the firewall blocks. An attacker on the internet cannot reach the physical workstation with no WAN access. From the office LAN, even a valid Windows password cannot open an RDP connection blocked by the firewall. At the portal, the technician must also supply a TOTP code.

If the portal is targeted, an attacker may relay a TOTP code in real time or hijack an existing session. Logs and recordings then help reconstruct what happened.

If the admin workstation is compromised, the attacker gains its position on the VLAN. Review the account’s permissions and the target server logs to establish what they could have done.

Sources and references

  1. UnitedHealth Group testimony before the US Congress ↩
  2. Apache Guacamole project ↩
  3. ANSSI — secure administration, chapters 4 and 8 ↩
  4. Apache Guacamole connection parameters ↩
  5. Microsoft — Remote Desktop Services ports ↩
  6. Apache — connection history ↩
  7. Apache — replaying recordings ↩
  8. GDPR, Articles 5 and 6 — purpose, proportionality and lawful basis ↩
  9. GDPR, Article 13 — informing participants ↩
  10. Apache — Guacamole architecture ↩
  11. Apache — TOTP extension ↩
  12. Apache — SAML extension ↩
  13. Microsoft — SAML authentication for an application ↩
  14. Microsoft — authentication strengths ↩
  15. Microsoft — Entra Conditional Access licensing ↩
  16. Apache Guacamole 1.6.0 — OpenID Connect extension implicit flow ↩
  17. Apache Guacamole 1.6.0 — terminating active sessions ↩
  18. Microsoft — separate on-premises and cloud emergency access ↩
  19. Truesec — investigation of FAUST ransomware ↩
  20. The DFIR Report — timeline and evidence of RDP connections ↩
  21. Check Point — RDP client research ↩
  22. Apache — advisories and patched versions ↩
  23. Debian — how APT verifies repositories and packages ↩
  24. Amazon — investigation of FortiGate access at scale ↩

My checklist for locking down server access