HTTP fundamentals, authentication mechanics, and API interaction — for engineers who want to understand web apps from the wire up.

Most of us interact with web applications through abstractions. We click buttons, fill forms, and trust that the browser is doing something sensible underneath. But when you’re assessing an application — whether for a pentest, a bug bounty, or just to understand a system you inherited — those abstractions become liabilities. You need to see the raw conversation between client and server.

HackTheBox’s Web Requests module is a compact, hands-on tour of that conversation. It walks through HTTP methods, authentication mechanisms, cookies, and REST-style APIs, all by having you replay and manipulate real requests using cURL and browser devtools. Here is what stuck with me.

GET vs. POST: Where Your Data Lives

The most useful mental model I took away is this: the HTTP method determines where your data lives, and that placement has consequences.

GET puts data in the URL (/search.php?search=le). It's logged, cached, and visible in browser history. It is fine for search terms, but terrible for sensitive data.

POST puts data in the body (username=admin&password=admin). When replaying a login request, the response included a Set-Cookie: PHPSESSID=... header. From then on, using curl -b 'PHPSESSID=...' authenticated me without a password. That’s also why session hijacking is so dangerous: a stolen cookie is a stolen identity.

One gotcha: the application’s search endpoint expected JSON ({"search":"london"}). Replaying it required -H 'Content-Type: application/json'. Without that header, the server parsed the body as form data, found no search= key, and failed silently. The body's format is meaningless without the label that declares it.

Basic Auth: Encoding Is Not Encryption

The unauthenticated request returned a 401 with WWW-Authenticate: Basic realm="Access denied". Supplying admin:admin via cURL produced this header:

Authorization: Basic YWRtaW46YWRtaW4=

That base64 string decodes to admin:admin. There is no cryptography here. It is simply a permission slip written in a language anyone can read. Basic auth is only safe over TLS, and even then, it should be treated as a minimum bar.

CRUD APIs: The Verb Is the Instruction

The final section reframed everything. The application exposed a clean API where the URL acted as the noun (/api.php/city/london), and the HTTP method acted as the verb:

• POST (Create)
• GET (Read)
• PUT (Update)
• DELETE (Delete)

The capstone exercise chained them together: rename a city to flag with PUT, delete another city with DELETE, then GET flag to retrieve the reward.

The real finding wasn’t the commands — it was what was missing. No cookie, no token, and no auth header were required for any of those operations. Anyone who could reach the endpoint could read, create, modify, or destroy data. That is Broken Access Control, a critical finding in a real assessment. The takeaway: when you find one working request, test every verb against the same URL. The gap between GET /resource and DELETE /resource is one word — and sometimes one missing authorization check.

Practical Takeaways

1. The Network tab is a specification. Clear it, perform the action, and read the request. Method, path, headers, cookies, and body format — that is the full contract. “Copy as cURL” is faster than reverse-engineering.
2. Cookies are credentials. Treat them with the same care as passwords.
3. Content-Type is load-bearing. Mismatches cause silent failures that look like server bugs.
4. Test every verb. A protected GET doesn't mean PUT and DELETE are.
5. jq is not optional. It turns unreadable JSON blobs into structures you can actually reason about.

Closing Thoughts

The module is short but dense. The progression — GET, POST, auth, cookies, JSON, CRUD — mirrors how apps are built and how assessments unfold: read the visible request, learn the contract, test the boundaries. Every concept came with a command you could run, which is why it sticks. Theory about Set-Cookie is abstract; watching your cookie authenticate you after you've logged out is not.

If you want to try these exercises yourself in a safe, authorized environment, you can find the module here on HackTheBox Academy.

All exercises were performed in HackTheBox’s authorized lab environment. The techniques described are for educational purposes and should only be applied to systems you have explicit permission to test.

A real-world case study on defeating Deep Packet Inspection (DPI) with obfs4 bridges and automating the entire process.

Living in a region with strict internet censorship is a daily exercise in frustration. In Myanmar, Internet Service Providers (ISPs) have deployed aggressive Deep Packet Inspection (DPI) technologies. They don’t just block specific websites; they actively hunt and throttle privacy-enhancing tools like the Tor network.

If you try to open the standard Tor browser, the connection is dropped instantly. The ISP sees the Tor handshake and slams the door shut.

I decided to build a reliable, system-wide censorship bypass on my Ubuntu 24.04 laptop. But instead of just following a standard tutorial, I had to engineer a solution that could survive the specific DPI blocks of my ISP.

Here is the story of how I used AnonSurf, obfs4 bridges, Bash, and Python to take my internet freedom back — and how you can do it too.

The Concept: Hiding in Plain Sight

To understand how we beat DPI, we have to understand how it works.

Standard Tor traffic is encrypted, but the initial “knock on the door” (the handshake) has a recognizable signature. DPI looks for this signature and blocks it.

To bypass this, we use Pluggable Transports, specifically obfs4. Think of obfs4 as a disguise. It takes your Tor traffic and scrambles it so that it looks like a normal, harmless HTTPS connection to a regular website. To the ISP, you are just reading the news. In reality, you are entering the Tor network.

But to connect to Tor using obfs4, you can’t use the public Tor entry nodes (they are all blocked). You must use Bridges — unlisted, secret entry points provided by the Tor Project.

Here is the architecture I built:

Diagram: How traffic flows from the laptop, through the obfs4 disguise, past the ISP’s DPI, and into the Tor network.

The Setup: AnonSurf and the Disguise

I chose AnonSurf to manage my connection. Unlike the Tor Browser (which only anonymizes the browser), AnonSurf uses iptables to route every single packet leaving my laptop through Tor. It prevents leaks from other apps like Telegram or my terminal.

1. I compiled AnonSurf from source for Ubuntu 24.04.
2. I requested obfs4 bridges using the Tor Project’s official Telegram bot (@GetBridgesBot).
3. I injected these bridges into AnonSurf’s configuration file.

Once connected, I verified my identity using check.torproject.org. Seeing {"IsTor":true} and a German IP address pop up on my screen was a massive win. I was officially invisible to my ISP.

Success! The ISP is bypassed, and traffic is routing through Europe.

Problem #1: The DNS Breakage (and the Bash Solution)

Everything worked perfectly until I tried to turn it off.

When I ran sudo anonsurf stop, my internet completely broke. I kept getting Temporary failure in name resolution.

I realized that AnonSurf hijacks the system’s DNS file (/etc/resolv.conf) to force all traffic through Tor. But when it stops, it fails to restore Ubuntu 24.04's default systemd-resolved symlink. It leaves the system in a broken state.

To fix this, I wrote a custom Bash automation script (anonsurf-toggle.sh).

Instead of just turning Tor on and off, this script acts as a manager. When I tell it to stop, it:

1. Safely kills AnonSurf.
2. Deletes the broken DNS file.
3. Recreates the correct system symlink.
4. Restarts the DNS service.
5. Pings Google to confirm the clearnet connection is restored.

Now, switching between my normal internet and the Tor network is a single, flawless command.

Problem #2: When the ISP Blocks the API (Enter Python)

To make the project portfolio-ready, I wanted to automate the most tedious part: fetching new bridges. Bridges get blocked frequently, so you constantly need fresh ones.

I wrote a Python script that queried the official Tor Project MOAT API, requested new obfs4 bridges, and automatically injected them into my configuration file.

There was just one problem: My ISP blocked the API.

When I ran the script, it threw a [Errno 101] Network is unreachable error. My ISP had successfully identified and blocked bridges.torproject.org via DPI.

But in Cybersecurity, failure isn’t the end; it’s just a prompt to pivot.

I updated the Python script to include graceful error handling. Instead of crashing, the script now specifically catches the ConnectionError. It prints a clear, human-readable message explaining that the ISP is blocking the API and automatically advises the user to use the Telegram Bot fallback method.

The Python script correctly identifying ISP DPI blocking and advising the fallback.

This highlights a crucial lesson in network security: APIs can be blocked, but humans will always find a backdoor. The Telegram bot cannot be easily blocked by the ISP without breaking normal communications, making it the ultimate fallback.

The Result: A Resilient Censorship Bypass

By combining Bash for system automation and Python for network resilience, I created a highly robust tool.

This project isn’t just about installing Tor; it’s about building a resilient infrastructure that survives a hostile network environment. It proved that while ISPs can block standard traffic and APIs, a combination of obfuscation (obfs4) and fallback planning can defeat them.

If you are facing censorship or want to see the exact code, scripts, and troubleshooting logs I used, I have open-sourced the entire project on GitHub.

👉 Check out the GitHub Repository here

Have you ever had to bypass DPI or fight ISP censorship? Let me know your strategies in the comments below!