2025-12-28 16:19:51
So far, we have used commands like:
apt updateapt upgradeapt dist-upgradeapt full-upgradeThese commands upgrade packages only, but they do NOT upgrade the Ubuntu OS version itself.
Ubuntu releases a new OS version every 6 months.
A release upgrade means upgrading from one Ubuntu version to another (for example: 22.04 → 22.10 → 23.04).
| Type | What it upgrades | Ubuntu version changes? |
|---|---|---|
apt upgrade |
Installed packages | ❌ No |
apt dist-upgrade |
Packages + dependencies | ❌ No |
do-release-upgrade |
Entire OS version | ✅ Yes |
A release upgrade is risky, especially on servers. Before upgrading, always check the following.
If the system becomes unbootable, the backup is your only recovery.
You need several GB of free space.
Check disk usage:
df -h
Example:
Unsupported repos cause:
This is a major risk factor.
This allows you to recover data if the OS fails.
Check your current version:
lsb_release -a
Example:
Ubuntu 22.04 LTS
Recommended for:
⚠️ Upgrading from LTS → non-LTS means losing long-term support
| System Type | Recommendation |
|---|---|
| Production server | Stay on LTS |
| Business-critical system | Stay on LTS |
| Workstation / testing | Optional |
| Learning / demo | Fine |
Always ask:
“What problem does this upgrade solve for me?”
sudo apt update
sudo apt full-upgrade
This ensures:
sudo reboot
Ensures new kernel is active.
sudo apt install update-manager-core
This provides:
do-release-upgradesudo do-release-upgrade
If no new LTS is available, you may see:
No new release found
Edit:
sudo nano /etc/update-manager/release-upgrades
Change:
Prompt=lts
to:
Prompt=normal
Save and exit.
sudo do-release-upgrade
Ubuntu will:
Examples:
Default choices are usually safe.
You may be asked:
Remove obsolete packages?
Example:
Configuration file '/etc/crontab' has been modified
Options:
Use D to inspect differences before deciding.
Kernel issues are critical because:
Causes:
sudo reboot
After reboot:
lsb_release -a
If successful, version is upgraded.
In this case:
This is realistic and valuable:
This is actually a perfect real-world example.
In real DevOps work:
This is far more valuable than a “happy-path” demo.
Symptoms:
This tells us:
Before touching anything:
Stress causes bad decisions.
Calm fixes systems.
Conclusion:
👉 Kernel is failing, not the bootloader.
Because GRUB was enabled earlier, we could:
Result:
This immediately isolates the issue.
If GRUB was hidden (default on many systems):
👉 Boot from a Live Linux system
Important rules:
Steps:
If errors appear:
Once live system is running:
You can browse:
/home/var/www/var/lib/mysql👉 Even if repair fails, your data is safe
This alone is a major win.
Before touching boot components, rule out disk corruption.
sudo fsck /dev/sda2
Result:
chroot
We need to work inside the broken system.
Right-click → Open in Terminal
sudo chroot .
What this does:
/ to the installed OSThis is critical for recovery.
Inside chroot, commands like:
update-grub
may fail with:
No such device
Why?
/dev, /proc, /sys are kernel-managedWe must bind system directories from the live kernel.
sudo mount --bind /dev /dev
sudo mount --bind /proc /proc
sudo mount --bind /sys /sys
Now the installed system can:
Now run:
update-grub
This time:
cat /boot/grub/grub.cfg
Find the exact menu entry of the working kernel.
Edit:
nano /etc/default/grub
Set:
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.19.x"
(Exact text must match grub.cfg)
update-grub
Exit chroot:
exit
Restart system:
reboot
Result:
We are not done yet.
Next steps:
👉 This is mandatory in production
Our system is booting again, but recovery is not finished yet.
A recovered system is still fragile unless we prevent the same failure from happening again.
Right now:
Common danger:
sudo apt autoremove
This command may silently remove old kernels if they are marked as auto-installed.
We must protect the working kernel.
Check the running kernel:
uname -r
Example:
5.19.0-xx-generic
Kernel files live in:
/boot
Important files:
vmlinuz-<version> → kernelinitrd.img-<version> → initial RAM diskThese files must not disappear.
Linux package manager knows which package created each file.
Check kernel ownership:
dpkg -S /boot/vmlinuz-5.19.0-xx-generic
Output example:
linux-image-5.19.0-xx-generic: /boot/vmlinuz-5.19.0-xx-generic
This tells us:
👉 The kernel comes from this package
This is the most important protection step.
sudo apt install linux-image-5.19.0-xx-generic
Why this works:
autoremove will never delete it
APT logic:
initrd.img Does Not Have a Package
You may notice:
dpkg -S /boot/initrd.img-5.19.0-xx-generic
returns nothing.
That is normal.
Reason:
initrd.img is generated dynamically
Verify:
sudo dpkg-reconfigure linux-image-5.19.0-xx-generic
You will see:
As long as the kernel package stays installed → initrd stays too.
If a newer kernel breaks boot, remove it.
dpkg -S /boot/vmlinuz-6.x.x-generic
sudo apt remove linux-headers-generic
sudo apt remove linux-image-6.x.x-generic
Why headers first?
⚠️ Do this only if you are sure the kernel is broken
linux-headers-generic Exists
This package:
Installing it later:
sudo apt install linux-headers-generic
Will:
Since the newest kernel caused failure:
👉 Do not reinstall it yet
Always test after kernel changes.
Reboot:
sudo reboot
Test:
If both work:
✅ System is stable again
Now that broken kernel is gone:
This is optional and not urgent.
Create test failures:
Practice makes incident response fast.
In real servers:
chroot only (no GUI)Same logic — just CLI only.
Even during rescue:
/home
/var
Never trust recovery until data is safe.
| Category | Examples |
|---|---|
| Kernel | incompatible kernel update |
| Bootloader | broken GRUB config |
| Filesystem | disk corruption |
| Packages | broken third-party drivers |
| Hardware | disk failure, overheating |
| Security | firewall blocks SSH |
| Mounts |
/etc/fstab errors |
Not all require live systems — boot failures do.
Before working with cron, you need to know one important thing:
👉 Cron is not one single program.
Historically, multiple cron implementations evolved independently.
They all look similar, but they may differ slightly in:
The concepts are the same, but details may vary.
The name comes from Chronos, the Greek word for time.
Cron reads multiple locations.
Stored internally in:
/var/spool/cron/crontabs/
Correct way to manage them:
crontab -e
Stored in:
/etc/crontab
Characteristics:
root
Used mainly for system-level tasks.
/etc/cron.d (Debian / Ubuntu)
This is Debian/Ubuntu-specific behavior.
crontab -e
EDITOR=vim crontab -e
or
EDITOR=nano crontab -e
crontab -l
This is the only safe way to read it without root access.
Even your own crontab:
/var/spool/cron/crontabs/<username>
Editing directly may:
👉 Always use crontab -e
A crontab has two parts:
These apply only to cron jobs, not your shell.
Common ones:
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Why this matters:
⚠️ Not all cron implementations support this
(works on Ubuntu/Debian)
MINUTE HOUR DAY MONTH DAY_OF_WEEK COMMAND
| Field | Range |
|---|---|
| Minute | 0–59 |
| Hour | 0–23 |
| Day | 1–31 |
| Month | 1–12 |
| Day of week | 0–7 (Sun=0 or 7) |
5 3 * * * command
*)
* means all possible values.
Example: every minute
* * * * * command
Cron runs without a terminal.
If you don’t redirect output:
Example:
* * * * * ping -c 1 google.com >> ~/ping.log
>> appendsAfter saving your crontab:
cat ~/ping.log
If output appears → cron works.
0 * * * * command
*/5 * * * * command
Runs at:
00, 05, 10, 15, 20, ...
0,15,30,45 * * * * command
0 8-20 * * * command
Both ends included.
0 */2 * * * command
⚠️ This runs every minute during matching hours if minute is *.
Correct way:
0 */2 * * * command
0 1-23/2 * * * command
Day of week acts as a filter.
Example: every Monday at midnight
0 0 * * 1 command
Values:
This:
* */2 * * * command
Means:
Result:
Often not what you want.
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
0 * * * * ping -c 1 google.com >> ~/ping_hourly.log
Runs:
cron
Used for:
Ubuntu:
CentOS:
Cron is used for:
Understanding:
is mandatory for DevOps and SysAdmins.
flock (Ubuntu)
Important
This lecture is Ubuntu-specific.
CentOS behaves differently and is covered in the next lecture.
So far, we always redirected output:
>> file.log
But what if:
👉 Cron tries to send the output by email to the job’s owner.
So initially:
Edit crontab:
crontab -e
Add:
* * * * * ping google.com
This runs every minute and produces output.
Wait one minute.
On Ubuntu, cron logs are handled by systemd.
Follow cron logs:
journalctl -u cron -f
You will see:
CRON: No mail transfer agent installed, discarding output
So:
Cron does not send mail itself.
It delegates email delivery to an MTA.
sudo apt install mailutils
This installs:
mail commandDuring install, choose:
General type of mail configuration: Local only
This means:
Finish installation.
Local emails are stored as plain text files:
/var/mail/<username>
Example:
sudo cat /var/mail/youruser
You will see:
👉 This is not internet email
👉 This is local system mail
Cron supports the MAILTO variable.
Edit crontab:
crontab -e
Add at the top:
Now cron will attempt to send output externally.
Even with MAILTO, emails may not arrive because:
Reconfigure postfix:
sudo dpkg-reconfigure postfix
Choose:
Internet Site
Accept defaults for:
This allows:
Your VM:
Result:
This is expected.
For production servers:
* * * * * command >> file.log 2>&1
/dev/null
* * * * * command > /dev/null 2>&1
# * * * * * command
flock Is Important (Real Production Problem)
Cron does not prevent overlap.
If a job runs every minute:
leads to:
flock Does
flock:
This allows mutual exclusion.
flock Example
Terminal 1:
flock /tmp/test.lock ping google.com
Terminal 2:
flock /tmp/test.lock ping google.com
Result:
flock (Cron-Safe)
Use:
flock -n /tmp/test.lock -c "command"
Behavior:
Cron logic:
0 = success!=0 = error → emailUsing:
flock -n file -c "cmd" || true
Ensures:
0 */2 * * * /usr/bin/flock -n /tmp/app.lock \
/usr/bin/php /var/www/app/artisan schedule:run
What this does:
Cron has a minimal PATH.
Best practice:
Find executable path:
which flock
which php
Without flock:
With flock:
Important
- This lecture is Ubuntu-specific
- CentOS / RHEL handle Anacron differently (covered separately)
- Concepts are shared, implementations differ
So far, we worked with user cron jobs:
crontab -e
Key points:
/var/spool/cron/crontabs/
crontab commandEven this:
sudo crontab -e
still creates a user cron job — this time for the root user.
/etc/crontab)
Ubuntu also supports system-wide cron jobs.
/etc/crontab
crontab -e)root
root
Security model:
/etc/crontab, they already have rootSo this is not a security risk.
Unlike user crontabs, one extra field exists.
MIN HOUR DAY MONTH DOW USER COMMAND
Example:
* * * * * alice echo "----" >> /home/alice/test.txt
This means:
alice
Edit the file:
sudo nano /etc/crontab
Add:
* * * * * alice cd /home/alice && echo "----" >> test.txt
After one minute:
ls /home/alice
Result:
test.txt existsalice
Example users:
www-datapostgresmysqlYou don’t want to log in as that user
Example:
*/5 * * * * www-data php /var/www/app/artisan cleanup
Regular cron jobs:
Anacron solves this.
Anacron:
Designed for:
Handles:
Typical use cases:
Use Anacron when:
Use cron when:
Ubuntu integrates Anacron via folders:
/etc/cron.daily/
/etc/cron.weekly/
/etc/cron.monthly/
How it works:
Allowed characters only:
_
-
Invalid filenames are ignored.
List daily jobs:
ls /etc/cron.daily
Example:
/etc/cron.daily/apache2
Open it:
sudo nano /etc/cron.daily/apache2
You’ll see a shell script:
Configuration file:
/etc/anacrontab
Format:
PERIOD DELAY JOB-ID COMMAND
Example:
1 5 cron.daily run-parts /etc/cron.daily
Meaning:
1 5 cron.daily
7 10 cron.weekly
@monthly 15 cron.monthly
This enables:
/etc/cron.daily/etc/cron.weekly/etc/cron.monthly/etc/cron.hourly Exists
Check /etc/crontab:
17 * * * * root cd / && run-parts --report /etc/cron.hourly
This is:
If the system is down → job is skipped.
/etc/crontab
You may see lines like:
25 6 * * * root test -x /usr/sbin/anacron || run-parts /etc/cron.daily
Meaning:
This guarantees:
Check Anacron’s systemd unit:
systemctl cat anacron.service
You will see:
ConditionACPower=true
Meaning:
To override:
sudo systemctl edit anacron.service
Add:
[Unit]
ConditionACPower=
Now:
Be aware:
When in doubt:
/etc/crontab allows per-user execution
The Internet is a network of networks.
This design makes the Internet:
Imagine:
You do not have a physical cable to Australia.
Instead:
This is called packet switching.
Important consequences:
Each router:
Routers are often called hops because packets “hop” through them.
To send data to google.com, several things must happen:
google.com → IP addressThis layered approach is intentional.
We will study networking from the ground up:
This matches how real networking works.
ip Command (Linux Networking)
ip?
The ip command is the modern Linux networking tool.
It replaces:
ifconfigroutenetstatIt is:
ip address show
Output shows:
Example interfaces:
lo → loopback (localhost)eth0, ens33, wlp0s20f3 → physical or virtual NICsifconfig -a
In this course, we use ip.
ip via Homebrew
macOS does not ship with ip.
Options:
ifconfig (native)ip wrapperbrew install iproute2mac
After installation:
ip address show
Notes:
Wireshark is a graphical packet analyzer.
It allows you to:
This is critical for understanding, not just memorizing.
Wireshark can:
Rules:
This lecture is for:
This is not legal advice.
sudo apt install wireshark
Start Wireshark with required privileges:
sudo wireshark
Steps:
Example:
google.com
http
Note:
Wireshark shows:
Right now this looks overwhelming — that’s expected.
By the end of this chapter:
The OSI (Open Systems Interconnection) model is a conceptual framework.
Purpose:
Developed:
Before standards:
Today:
That did not happen by accident.
| Layer | Name | Purpose |
|---|---|---|
| 1 | Physical | Bits on wire (cables, signals) |
| 2 | Data Link | Local delivery (MAC, Ethernet) |
| 3 | Network | Routing between networks (IP) |
| 4 | Transport | Reliability, ordering (TCP/UDP) |
| 5 | Session | Session management |
| 6 | Presentation | Encryption, compression |
| 7 | Application | HTTP, SSH, FTP, SMTP |
This is how real troubleshooting is done.
Each layer can evolve independently.
Example:
Devices from different vendors work together.
You can say:
This saves hours in production.
The physical layer (Layer 1) is the foundation of networking.
It is responsible for physically transmitting bits from one device to another.
This includes:
At this layer, there is no concept of IP addresses, packets, or routing—only raw bits.
Layer 1 handles:
Physical media
Signal transmission
Bit encoding
Timing & synchronization
Collision avoidance (basic mechanisms)
Basic error detection
Important detail:
Common Layer 1 problems:
If Layer 1 fails, nothing above it can work.
Old Ethernet splitters literally connected wires together.
All devices shared the same electrical medium.
You cannot unplug a cable with software.
But you can:
This effectively shuts down Layer 1 from the OS perspective.
ip addr show
Example interface names:
enp0s5 (modern, predictable)eth0 (older style)wlan0 (Wi-Fi)Modern names are stable and tied to hardware location.
sudo ip link set dev enp0s5 down
Result:
⚠️ Warning
If this interface is your SSH connection → you will disconnect immediately.
sudo ip link set dev enp0s5 up
Connectivity returns.
On systems like:
If you disable:
wlan0 (Wi-Fi)eth0 (Ethernet)Your remote session will drop.
Recovery requires:
This is a classic Layer 1 outage.
The Data Link Layer (Layer 2) handles local communication inside one network.
Key responsibilities:
Layer 2 does not route between networks.
Note:
- A switch is hardware
- A bridge is usually software
- Functionally, they are similar
Switch / Access Point
Router
A Wi-Fi access point is essentially:
A Layer-2 switch with radio antennas
Works for:
Fails for:
A switch learns MAC addresses.
Switch remembers:
Frames are forwarded only where needed
Example:
With a switch:
This is why switches replaced hubs.
From the computer’s perspective:
It behaves as if:
The switch is completely transparent.
This fact is critical when later learning about:
That is Layer 3.
| Layer | Purpose | Example |
|---|---|---|
| Layer 1 | Physical transmission | Cable, Wi-Fi |
| Layer 2 | Local delivery | Switch, MAC |
On Layer 2 (Data Link), we learned:
➡️ Problem
If two computers are not in the same network, Layer 2 is not enough.
That is why we need Layer 3 – the Network Layer.
| Layer | Unit | Address Type | Scope |
|---|---|---|---|
| Layer 2 | Frame | MAC address | Local network only |
| Layer 3 | Packet | IP address | Across networks |
Routing = forwarding data between networks
When sending data:
At every router:
This happens extremely fast in hardware.
A network is a group of interconnected devices that can communicate.
➡️ The Internet is a WAN made of many LANs
A router:
A default gateway:
ip addr show
You will see:
Example:
192.168.1.23/24
ip route show
Example:
default via 192.168.1.1 dev enp0s5
Meaning:
➡️ Same IP protocol
➡️ Different Layer-2 destination
| Destination | Ethernet Frame Goes To |
|---|---|
| Same network | Target device MAC |
| Different network | Router MAC |
This decision is made using the subnet mask.
A subnet is a logical subdivision of a network.
Used to:
Your computer must answer:
Is the destination IP local or remote?
If local → send frame directly
If remote → send frame to gateway
Example:
IP address: 192.168.1.10
Subnet mask: 255.255.255.0
CIDR: /24
Subnet mask:
Subnet mask has:
Logical AND:
If:
(network part of source)
==
(network part of destination)
➡️ Same subnet
Otherwise ➡️ send to gateway
Computers do this instantly in hardware.
Instead of:
255.255.255.0
We write:
/24
Why?
Examples:
| CIDR | Hosts (usable) |
|---|---|
| /24 | 254 |
| /23 | 510 |
| /22 | 1022 |
| /30 | 2 (point-to-point) |
For /24:
.0 → network address.255 → broadcast address.1 – .254 → usable hostsip addr show
Example output:
inet 192.168.1.23/24
This tells you:
We know IP packets contain destination IPs
We know Ethernet frames need destination MACs❓ How does the system know which MAC address to use?
That is the job of ARP.
| Layer | Unit | Address Used |
|---|---|---|
| Layer 3 | Packet | IP address |
| Layer 2 | Frame | MAC address |
To send any IP packet, the system must:
ARP answers one question only:
“Which MAC address owns this IP address?”
ARP works only inside the local network.
ping 192.168.1.50
Who has 192.168.1.50?
192.168.1.50 is at AA:BB:CC:DD:EE:FF
ping google.com
142.250.x.x) Who has 192.168.1.1?
➡️ IP destination stays Google
➡️ MAC destination is router
ARP traffic is normal and frequent:
In Wireshark:
ARP Who has 192.168.1.X?
ARP 192.168.1.X is at …
This is normal network noise, not a problem.
ip addr show
sudo ip addr add 192.168.1.232/24 dev enp0s5
Now the interface has two IPs.
✔️ Other devices in the same subnet can reach it
❌ Devices outside the subnet will not
sudo ip addr del 192.168.1.232/24 dev enp0s5
You can add:
8.8.8.8/24
But:
➡️ IP must match subnet logic, not just syntax.
ip route show
Typical output:
192.168.1.0/24 dev enp0s5
default via 192.168.1.1 dev enp0s5
Meaning:
ip route get 8.8.8.8
or
ip route get 192.168.1.50
You will see:
sudo ip route add 9.9.9.9/32 via 192.168.1.100 dev enp0s5
Result:
Remove it:
sudo ip route del 9.9.9.9/32
Example:
192.168.1.0/24
192.168.2.0/24
Without route:
With route:
sudo ip route add 192.168.2.0/24 via 192.168.1.5 dev enp0s5
Now traffic flows.
Dynamic Host Configuration Protocol
Automatically assigns:
Usually runs on the router.
All initial messages are broadcast.
Filter:
dhcp
You’ll see:
This is your entire network configuration being delivered.
journalctl -u systemd-networkd
Look for:
If:
➡️ It’s almost always DHCP
To send data:
Not all Linux distributions manage networking the same way.
Both tools:
They just do it differently.
NetworkManager is a more integrated solution.
It supports:
With systemd:
Networking is split across multiple components
systemd-networkdsystemd-resolved➡️ Recommendation
Use the default tool provided by your distribution unless you have a strong reason to change it.
On systems using NetworkManager (e.g. CentOS):
sudo journalctl -u NetworkManager --boot
What you’ll see:
Key events you’ll notice:
DHCP leases expire unless renewed.
The client periodically tells the router:
“I’m still here. Please keep my IP.”
This prevents:
Regardless of tool:
If:
➡️ Check DHCP logs first
ping is a Layer-3 diagnostic tool.
It uses ICMP (Internet Control Message Protocol).
If ping fails, it does NOT always mean:
It may mean:
Ping tests reachability, not availability.
Filter:
icmp
You will see:
Round-trip time (RTT):
Ping helps answer:
Ping does not:
Traceroute shows:
Command:
traceroute google.com
Output:
Network latency fluctuates.
Traceroute sends multiple probes to:
Typical observations:
* * * means:
When tracing overseas destinations:
Caused by:
This is why:
Every IP packet has:
TTL – Time To Live
Purpose:
➡️ Each step discovers one hop
When a router replies:
That’s why Wireshark still matches filters.
Each hop is:
Traceroute assumes:
Traceroute can show:
Example:
Traceroute exposes this.
You now understand:
At this point, you have solid Layer-3 knowledge, exactly what:
must understand deeply.
So far, Layer 3 (IP) solved routing across networks.
But IP alone has serious limitations:
Routers intentionally drop packets when overloaded — this is normal and expected.
➡️ Layer 4 exists to handle these problems
UDP (User Datagram Protocol):
Because sometimes retransmission is worse than packet loss.
Examples:
If a video frame arrives late → it’s already useless
➡️ Better to drop it and move on
Applications using UDP usually:
TCP (Transmission Control Protocol) provides:
Applications see TCP as a continuous stream, not packets.
➡️ Applications don’t need to care about packet loss.
Each TCP segment contains:
Used to:
TCP does not count packets — it counts bytes.
Before data transfer, TCP builds a connection:
Client → Server
Server → Client
Client → Server
➡️ Connection is now established
After this:
When using tools like wget or a browser:
You will see:
Followed by normal data packets
This knowledge is critical for:
Ports are Layer 4 identifiers.
0 – 65535
A connection is uniquely identified by:
Source IP + Source Port + Destination IP + Destination Port
Require root privileges.
Examples:
Assigned to common services.
Examples:
Used by clients:
Example:
Server response:
This allows thousands of simultaneous connections.
Why UDP here?
Port scanning tries to:
⚠️ Only scan systems you own or are authorized to scan
Laws vary by country — never assume legality
Install:
sudo apt install nmap
# or
sudo dnf install nmap
Basic scan:
sudo nmap localhost
Scans:
sudo nmap -p 22,80,443 192.168.1.10
sudo nmap -p- 192.168.1.10
sudo nmap 192.168.1.1-100
Useful for:
Port scanning helps you:
Example:
sudo systemctl stop mysql
sudo systemctl disable mysql
Re-scan:
sudo nmap localhost
➡️ Security improved
You now understand:
This knowledge is mandatory for:
Not all port scans behave the same way.
Scan type directly affects:
This is why an Nmap introduction is incomplete without scan types.
-sS) — Stealth Scan
| Response | Meaning |
|---|---|
| SYN-ACK | Port open |
| RST | Port closed |
| No reply | Port filtered (firewall) |
sudo nmap -sS localhost
➡️ Default and preferred scan if available
-sT) — Full Connection Scan
When SYN scan is not possible
Performs full TCP handshake
Uses OS networking stack
nmap -sT localhost
➡️ High visibility scan — use carefully
-sU) — Slow but Necessary
| Response | Meaning |
|---|---|
| UDP reply | Port open |
| ICMP error | Port closed |
| No reply | Open or filtered |
sudo nmap -sU localhost
But still critical because:
➡️ TCP scans alone are not enough
Nmap answers:
Security hardening workflow:
192.168.0.0/1610.0.0.0/8172.16.0.0/12Not routable on the Internet.
➡️ Router maintains a NAT table
Incoming traffic:
➡️ NAT works outbound only
Example:
:80
192.168.1.50:8080
Router rewrites:
Problem:
Solution:
Always reserve IPs for:
Most ISPs:
Solution:
Router updates DNS record automatically:
myhome.ddns-provider.com → current public IP
⚠️ Not production-grade
Adds:
Modern reality:
Emails require:
MIME defines:
➡️ Transport (SMTP) ≠ Representation (MIME)
Important truth:
Example:
HTTP/3 (QUIC)
➡️ One protocol can span multiple OSI layers
You now understand:
This knowledge is essential for:
The application layer (Layer 7) consists of protocols used by applications, not the applications themselves.
Example:
According to the OSI model, the protocol is Layer 7, not the program you click.
| Protocol | Purpose |
|---|---|
| HTTP / HTTPS | Web access |
| IMAP | Access emails on server |
| POP3 | Download emails |
| SMTP | Send emails |
| SSH | Remote shell, file transfer |
| FTP / SFTP | File transfer |
| DNS | Name resolution |
| Custom APIs | REST, gRPC, proprietary |
Layer 7 protocols depend on all lower layers:
DNS is an application-layer protocol that converts human-readable names into IP addresses.
Example:
google.com → 142.250.x.x
Humans remember names better than IP addresses.
Computers require IP addresses to communicate.
DNS bridges that gap.
The browser checks:
If yes → use cached IP.
If browser cache misses:
If found → return IP.
If OS cache misses:
Resolvers also cache results heavily.
.com
Response:
Ask the .com TLD servers
.com TLD serversAsk Google’s authoritative name servers
ns1.google.com
google.com → IP addresses
✅ DNS resolution complete.
| Record | Purpose |
|---|---|
| A | Domain → IPv4 |
| AAAA | Domain → IPv6 |
| CNAME | Alias to another domain |
| MX | Mail servers |
| NS | Authoritative name servers |
| TXT | Verification, SPF, DKIM, metadata |
host -a google.com
Example output:
Using dig shows exactly what DNS is doing.
dig @a.root-servers.net com NS
Returns:
.com TLD serversdig @a.gtld-servers.net google.com NS
Returns:
dig @ns1.google.com google.com A
Returns:
This demonstrates DNS hierarchy and delegation clearly.
DNS was designed before modern security threats.
Even if DNS is spoofed:
This mitigates DNS attacks, though it does not fix DNS itself.
/etc/hosts — Manual DNS Override
File:
/etc/hosts
Format:
IP_ADDRESS hostname
Example:
127.0.0.1 myproject.local
/etc/hosts
⚠️ Overrides DNS entirely
sudo nano /etc/hosts
Add:
127.0.0.1 myproject.local
Test:
ping myproject.local
127.0.0.1 google.com
Result:
Useful only for testing or demos.
Sometimes /etc/hosts changes don’t apply immediately.
Reason:
sudo lsof -i :53
Common:
systemd-resolveddnsmasqsudo resolvectl flush-caches
Verify:
resolvectl statistics
sudo systemctl restart dnsmasq
You now understand:
dig
/etc/hosts overridesThis knowledge is critical for:
A hostname is a human-readable name assigned to a computer on a network.
hostname
Example output:
ubuntu
Some shells show it automatically in the prompt, but this is configurable.
From another machine:
ping ubuntu
ping ubuntu.local
If hostname resolution is configured correctly, the hostname resolves to an IP.
/etc/hostname
sudo nano /etc/hostname
Example:
vm-ubuntu
sudo reboot
After reboot:
hostname
Output:
vm-ubuntu
/etc/hosts Must Also Be Updated
The hostname should resolve locally to the loopback interface.
Edit:
sudo nano /etc/hosts
Correct example:
127.0.1.1 vm-ubuntu
127.0.0.1 localhost
127.0.1.1?
localhost
Best practice: always update /etc/hosts after hostname change
.local Hostnames and mDNS
.local?
.local is a reserved domain for multicast DNS (mDNS).
It ensures:
.local Is Required
❌ Bad:
server.london
✔ Good:
server.london.local
.local guarantees the name never escapes your LAN.
Who is raspberrypi.local?
I am raspberrypi.local → 192.168.1.29
No central DNS server required.
| OS | Implementation |
|---|---|
| macOS | Bonjour / Zeroconf |
| Linux | Avahi |
| Windows | Partial support |
| Routers | Often integrated |
On some distributions (e.g. CentOS):
sudo dnf install nss-mdns
sudo reboot
Without this:
mdns
You will see:
✔ Always use:
hostname.local
✔ Or use static IPs if stability is critical
✔ Avoid bare hostnames without .local
Example request:
GET / HTTP/1.1
Host: www.google.com
User-Agent: Firefox
Accept: text/html
HTTP/1.1 200 OK
Content-Type: text/html
Content-Encoding: br
Then:
telnet www.google.com 80
GET / HTTP/1.1
Host: www.google.com
(blank line required)
Example malformed request:
HELLO WORLD HTTP/9.9
Expected result:
400 Bad Request
A good server never crashes.
Example:
192.168.1.10
Example:
2001:db8::1
Full:
2001:0db8:0000:0000:0000:0000:0000:0001
Shortened:
2001:db8::1
Rules:
:: only once✔ No NAT
✔ Easier routing
✔ Every device gets public IP
✔ Firewalls replace NAT for security
✔ IPv4 + IPv6 enabled
✔ Servers reachable on IPv4
✔ IPv6 preferred internally
✔ Fallback always works
| Scenario | Recommendation |
|---|---|
| Internal networks | Dual stack |
| Public servers | IPv4 mandatory |
| Future-proofing | Add IPv6 |
| Debugging | Test both stacks |
You’ll see:
IPv6 ≠ exotic — it’s already active.
✔ Hostnames simplify local networking
✔ Always update /etc/hosts
✔ Use .local for LAN resolution
✔ mDNS uses multicast (not DNS)
✔ HTTP is plain text over TCP
✔ Telnet is a powerful debug tool
✔ IPv6 removes NAT limitations
✔ IPv4 still required today
SSH (Secure Shell) is a cryptographic network protocol used to securely access and manage remote systems over a network.
SSH provides:
SSH is one of the most important tools for Linux, DevOps, Cloud, and Security engineers.
scp (Secure Copy)sftp (SSH File Transfer Protocol)In this course, we focus on shell access, file transfer, and security basics.
SSH always consists of two components:
sshd)
ssh)
Everything you practice here applies directly to:
You need two machines that can reach each other.
Pros
Cons
Pros
Cons
Key steps:
ip addr show
ping <other_vm_ip>
ping ubuntu.local
If ping works → SSH will work.
ip addr show
You should see:
192.168.x.x
ping 192.168.x.x
ping ubuntu.local
On the machine you want to control:
sudo apt update
sudo apt install openssh-server
Verify service:
systemctl status ssh
SSH server starts automatically.
ssh username@host
Examples:
ssh [email protected]
ssh [email protected]
ssh [email protected]
If username is omitted:
ssh host
SSH uses your local username by default.
You may see:
The authenticity of host cannot be established.
This is normal.
Type:
yes
This stores the server’s host key fingerprint.
We will cover this security mechanism in detail later.
Once connected:
exit closes the connection
SSH is encrypted, but exposure still matters.
Bad:
sanfrancisco
Good:
A9$eP7!xQm
Edit config:
sudo nano /etc/ssh/sshd_config
Change:
Port 22
To:
Port 2222
Save file.
Always validate before restart:
sudo sshd -t
If no output → config is valid.
Restart service:
sudo systemctl restart ssh
Existing sessions remain active.
ssh -p 2222 user@host
Example:
ssh -p 2222 [email protected]
Some networks block uncommon ports:
Solutions
/var/log/auth.log
/var/log/secure
View SSH activity:
grep sshd /var/log/auth.log
You will see:
Changing the SSH port makes real attacks visible, not buried in noise.
SSH is:
Understanding SSH deeply is non-negotiable for:
By default:
All local users with passwords can SSH
This is not ideal.
In sshd_config:
AllowUsers yannis
Multiple users:
AllowUsers yannis deploy admin
Validate & restart:
sudo sshd -t
sudo systemctl restart sshd
If you mistype the username:
Always keep one SSH session open.
Why?
sudo systemctl stop sshd
You can fix it:
sudo systemctl start sshd
This prevents rescue-mode recovery.
Passwords are:
SSH keys solve all of this.
ssh-keygen -t rsa -b 4096
Press Enter for defaults.
Files created:
~/.ssh/id_rsa (PRIVATE – never share)
~/.ssh/id_rsa.pub (PUBLIC – safe to share)
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 2222 user@server
Enter your password once.
ssh -p 2222 user@server
✔ No password
✔ Secure
✔ Perfect for automation
Location:
~/.ssh/authorized_keys
Permissions:
~/.ssh → 700
authorized_keys → 600
Each line = one allowed public key
Comments help identify owners.
SSH keys are not optional in real environments.
Now that public/private key authentication is configured, allowing password login is unnecessary and risky.
sudo (privilege escalation)sudo <command>
This means:
SSH key ≠ root access
From your client:
ssh -p 2222 user@server
You should log in without a password prompt.
⚠️ If this does not work, STOP. Do not continue.
Edit SSH server configuration:
sudo nano /etc/ssh/sshd_config
Set:
PasswordAuthentication no
(Optional but recommended)
PermitEmptyPasswords no
sudo sshd -t
No output = configuration is valid
Restart SSH:
sudo systemctl restart sshd
Switch to a user without SSH keys (or another local user):
ssh -p 2222 user@server
Expected result:
Permission denied (publickey).
✔ Password login is now fully disabled
✔ Only authorized SSH keys can log in
If teammates still use passwords:
If your laptop is:
You cannot log in.
authorized_keys
If someone gets your private key:
SSH connections may drop if:
This is annoying and dangerous:
SSH can send empty packets periodically to keep the connection alive.
Best practice:
Configure this on the client, not the server.
Edit user SSH config:
nano ~/.ssh/config
Add:
Host *
ServerAliveInterval 60
ServerAliveCountMax 3
chmod 600 ~/.ssh/config
As long as:
Your SSH session remains active.
These features prevent:
This is mandatory knowledge for:
sshd is installedWhen you connect for the first time, SSH asks:
“Are you sure you want to continue connecting?”
Once accepted, the fingerprint is saved locally.
~/.ssh/known_hosts
This file maps:
hostname → fingerprint
From that moment on:
If SSH says:
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
Possible causes:
Instead of connecting directly:
You → Attacker → Real Server
The attacker:
SSH fingerprints are what detect this attack.
SSH traffic is encrypted, but:
Fingerprint verification proves the server identity.
On the server:
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
(Or RSA if used)
SSH shows:
SHA256:QOG...ELkww
Both must match exactly.
Only then should you type:
yes
If SSH is your only access:
After first trust:
SSH includes:
(Some servers allow SFTP only, no shell)
In file manager:
sftp://user@hostname
scp user@server:/home/user/file.txt .
scp -r user@server:/home/user/folder .
scp file.txt user@server:/home/user/
scp -P 2222 file.txt user@server:/home/user/
⚠️ SCP uses uppercase -P
⚠️ SSH uses lowercase -p
Cyberduck supports:
Steps:
screen: Shared & Persistent Terminal Sessions
screen?
screen
screen
# Ubuntu
sudo apt install screen
# CentOS
sudo dnf install screen
screen Workflow
screen
Ctrl + A, then Ctrl + D
screen -ls
screen -x <session-id>
screen
screen -x
Now:
| Action | Effect |
|---|---|
exit |
Terminates the session |
Ctrl+A Ctrl+D |
Detaches safely |
To fully stop screen:
exit
exit
(Exit shell → exit screen)
screen Belongs in Your Toolbox
screen enables shared terminalsThis completes a professional, real-world SSH workflow used daily by DevOps engineers.
2025-12-28 16:04:57
Project: Exploratory Data Analysis on Restaurant Tips Dataset
Duration: Full EDA Process
Dataset: 243 restaurant transactions, 7 variables
Status: ✅ COMPLETED
total_bill, tip, size
sex, smoker, day, time
Understand what factors influence tipping behavior in restaurants through comprehensive exploratory data analysis.
Hypothesis: "The Null Hypothesis" - Why might data be missing?
What I Did:
# Checked for missing values
data.isnull().sum()
data.isnull().any()
(data.isnull().sum() / len(data)) * 100 # Percentage
Results:
Learning Moment: Not all datasets have missing data, but always check!
Hypothesis: Could identical transactions exist legitimately?
What I Did:
# Found duplicates
num_duplicates = data.duplicated().sum()
duplicates = data[data.duplicated(keep=False)]
# Removed them
data_clean = data.drop_duplicates()
Results:
Key Insight: Identical transactions on same day/time are statistically improbable - likely errors.
Hypothesis: "The Outlier Tribunal" - Are extreme values errors or legitimate?
What I Did:
# Created boxplots
plt.boxplot(data['tip'])
plt.boxplot(data['total_bill'])
# Calculated IQR boundaries
Q1 = data['tip'].quantile(0.25)
Q3 = data['tip'].quantile(0.75)
IQR = Q3 - Q1
upper_boundary = Q3 + (1.5 * IQR)
# Found outliers
outliers = data[data['tip'] > upper_boundary]
Mathematical Formula:
IQR = Q3 - Q1
Upper Boundary = Q3 + (1.5 × IQR)
Lower Boundary = Q1 - (1.5 × IQR)
For Tips:
Q1 = $2.00
Q3 = $3.56
IQR = $1.56
Upper Boundary = $5.90
Outliers Found:
| Bill | Tip | Tip % | Verdict |
|---------|--------|-------|-----------------|
| $50.81 | $10.00 | 19.7% | ✅ Legitimate |
| $48.33 | $9.00 | 18.6% | ✅ Legitimate |
| $39.42 | $7.58 | 19.2% | ✅ Legitimate |
| $48.27 | $6.73 | 13.9% | ✅ Legitimate |
Decision: Kept all outliers - they represent large parties with reasonable tip percentages
Key Insight: Outliers aren't always errors! Verify with context (tip percentage in this case).
For each relationship, I followed the scientific method:
My Hypothesis:
What I Did:
plt.scatter(data_clean['total_bill'], data_clean['tip'])
plt.xlabel('Total Bill ($)')
plt.ylabel('Tip ($)')
plt.title('Relationship Between Total Bill and Tip Amount')
plt.show()
Results:
Hypothesis Verdict: ❌ REJECTED
What I Learned:
Business Insight: Higher bills = higher tips. Restaurants should encourage higher spending.
My Hypothesis:
What I Did:
plt.scatter(data_clean['size'], data_clean['total_bill'])
Results:
Hypothesis Verdict: ✅ CONFIRMED (but weaker than expected)
Key Insight:
My Hypothesis:
Results:
Hypothesis Verdict: ⚠️ PARTIALLY CORRECT
Surprising Discovery:
Possible Explanations:
Key Insight: Large parties tip differently than expected - real behavioral economics!
My Hypothesis:
What I Did:
sns.boxplot(x='day', y='tip', data=data_clean,
order=['Sun','Mon','Tue','Wed','Thur','Fri','Sat'])
Results:
| Day | Avg Tip | Verdict |
|-----------|---------|----------------|
| Saturday | $3.00 | 🏆 Highest |
| Sunday | $2.90 | High |
| Mon/Tue/Wed| $2.25 | 🔻 Lowest (tie)|
Hypothesis Verdict: ⚠️ PARTIALLY WRONG
What I Got Wrong:
Key Observations:
Key Insight:
"Sunday = diverse people = diverse tipping = large variation in tips"
My Hypothesis:
Results:
| Time | Avg Tip | Difference |
|--------|---------|------------|
| Dinner | $3.00 | — |
| Lunch | $2.20 | -$0.80 |
Hypothesis Verdict: ✅ CONFIRMED!
Key Insight:
Business Recommendation: Prioritize dinner service quality!
My Hypothesis:
Results:
| Sex | Avg Tip | Difference |
|--------|---------|------------|
| Female | $3.20 | — |
| Male | $3.00 | -$0.20 |
Hypothesis Verdict: ❌ REJECTED!
What I Got Wrong:
Key Insight: Gender stereotypes about tipping don't hold up in data!
My Hypothesis:
Results:
| Smoker Status | Avg Tip | Difference |
|---------------|---------|------------|
| Smokers | $3.00 | — |
| Non-smokers | $2.80 | -$0.20 |
Hypothesis Verdict: ❌ REJECTED!
Honest Reflection: "Cannot figure out why" - and that's okay!
Possible Explanations:
Key Insight: Not every pattern has an obvious explanation - intellectual honesty matters!
# Correlation matrix
correlation_matrix = data_clean[['total_bill', 'tip', 'size']].corr()
sns.heatmap(correlation_matrix, annot=True, cmap='coolwarm')
| Pair | Correlation | Strength | Interpretation |
|---|---|---|---|
| total_bill ↔ tip | 0.67 | Strong | 🏆 Strongest predictor |
| size ↔ total_bill | 0.60 | Medium-Strong | More people = more food |
| size ↔ tip | 0.49 | Medium-Weak | Non-linear (plateaus) |
Key Insight:
"Tip percentage is fixed as that of total bill" - this explains the 0.67 correlation perfectly!
sns.pairplot(data_clean,
vars=['total_bill', 'tip', 'size'],
hue='time', # Color by lunch/dinner
diag_kind='hist')
From diagonal (distributions):
From scatter plots:
Overall Impression: Relationships are SOMEWHAT CLEAR - not perfect, but strong enough to be meaningful
Total Bill (r=0.67) 🥇
Time of Day ($0.80 difference) 🥈
Party Size (r=0.49 with tip) 🥉
Day of Week ($0.75 difference)
Sex ($0.20 difference) - WEAK
Smoker Status ($0.20 difference) - WEAK
Based on data analysis, restaurant owners should:
Why: Strongest correlation (0.67) - higher bills directly lead to higher tips
Actions:
Expected Impact: 10% increase in average bill → ~10% increase in tips
Why: Dinner tips $0.80 (36%) more than lunch
Actions:
Expected Impact: Shift focus to higher-margin time period
Why: These sizes have best tip-to-effort ratio
Actions:
Expected Impact: Maximize tips per table/server time
Why: Saturday/Sunday have higher tips
Actions:
Why: These factors have minimal effect
Insight: Treat all customers equally - demographics don't significantly predict tipping
My Wrong Predictions:
Lesson: Don't trust assumptions - test with data!
import pandas as pd # Data manipulation
import numpy as np # Numerical operations
import matplotlib.pyplot as plt # Basic plotting
import seaborn as sns # Statistical plotting
Pandas:
data.head() # First 5 rows
data.shape # Dimensions
data.describe() # Statistics
data.isnull().sum() # Missing values
data.duplicated().sum() # Duplicates
data.drop_duplicates() # Remove duplicates
data['column'] # Select column
data[condition] # Filter rows
data.groupby().mean() # Group and aggregate
data.corr() # Correlation matrix
Matplotlib:
plt.scatter(x, y) # Scatter plot
plt.xlabel() # X-axis label
plt.ylabel() # Y-axis label
plt.title() # Title
plt.grid() # Grid lines
plt.show() # Display plot
Seaborn:
sns.boxplot(x='category', y='number', data=df)
sns.heatmap(corr_matrix, annot=True)
sns.pairplot(df, vars=['col1','col2'], hue='category')
1. Order Matters!
# WRONG
outliers = data[data['tip'] > 5]
data['tip_pct'] = data['tip'] / data['total_bill']
print(outliers['tip_pct']) # ERROR!
# RIGHT
data['tip_pct'] = data['tip'] / data['total_bill']
outliers = data[data['tip'] > 5]
print(outliers['tip_pct']) # WORKS!
2. Matplotlib vs Seaborn:
3. Data Types Matter:
✅ Technical Skills:
✅ Analytical Skills:
✅ Soft Skills:
Total: 9 professional visualizations
"Party size plateaus at 5-6 people!"
"Total amount of spending is the determining factor"
End of Journey Summary
"The only real mistake is the one from which we learn nothing." - Henry Ford
"In God we trust. All others must bring data." - W. Edwards Deming
2025-12-28 16:03:03
When developing web applications, we often encounter challenges related to session management or tracking. Most of these issues trace back to a fundamental understanding of Cookies.
To solve these problems, it is essential to revisit the basics of how cookies work. In this series, I will organize the fundamentals of cookies, specifically tailored for engineers. In this first part, we will cover the definition of cookies and the crucial distinction between 1st Party and 3rd Party Cookies.
A Cookie is a small piece of data stored in the user's browser.
Since HTTP is a "stateless" protocol—meaning each request is independent and the server doesn't remember previous interactions—cookies play a vital role in maintaining state.
One of the most important concepts to understand in modern web development is the difference between these two types.
example.com, any cookie issued by example.com is a 1st party cookie.example.com, an ad network script from ad-network.net issues its own cookie.In the context of Risk-Based Authentication (RBA) or security design, understanding this difference is mandatory. You need to know:
To build robust systems, you should aim for designs that rely on secure 1st Party Cookies rather than unstable 3rd party ones.
Understanding these basics is the prerequisite for diving into implementation and browser-specific behaviors.
In Part 2, we will explore "Browser Cookie Behavior" in more detail, including:
2025-12-28 16:00:26
Most AI interview platforms are glorified chatbots with better questions. We built Squrrel to do something harder: have actual spoken conversations with candidates.
That decision nearly killed the product before launch.
The Obvious Choice That Wasn't Obvious
When I started building Squrrel, the safe play was text-based interviews. Lower latency, fewer technical headaches, easier to parse and analyze. Every AI product manager I talked to said the same thing: "Start with chat. Voice is a nightmare."
They were right about the nightmare part.
But I kept coming back to one fact: 78% of recruiting happens over the phone. Not email. Not Slack. Phone calls. Because hiring managers want to hear how candidates think on their feet, how they structure explanations, whether they can articulate complex ideas clearly.
A text-based interview platform would be easier to build and completely miss the point.
So we went with voice. And immediately discovered why everyone warned us against it.
The Technical Debt I Didn't See Coming
Speech recognition for interviews is different from speech recognition for everything else.
Siri and Alexa are optimized for short commands. Transcription tools like Otter are optimized for meetings with multiple speakers. We needed something that could handle:
20-40 minute monologues about technical projects
Industry jargon that doesn't exist in standard training data ("Kubernetes," "PostgreSQL," "JWT authentication")
Non-native English speakers with varying accents
Candidates who talk fast when nervous or slow when thinking
Off-the-shelf speech-to-text models failed spectacularly. Our first pilot had a 23% word error rate on technical terms. A candidate said "I implemented Redis caching" and got transcribed as "I implemented ready's catching." Recruiters couldn't trust the output.
I spent three weeks fine-tuning Wav2Vec 2.0 on domain-specific data—transcripts from actual tech interviews, recordings of engineers explaining their work, podcasts about software development. Got the error rate down to 6% for technical vocabulary.
But here's what surprised me: the remaining errors weren't random. They clustered around moments of hesitation, filler words, and self-corrections—exactly the moments that reveal how someone thinks under pressure.
We almost removed those "errors" before realizing they were features, not bugs.
The Conversational AI Problem Nobody Talks About
Building an AI that can conduct a natural interview conversation is way harder than building one that asks scripted questions.
The models are good at turn-taking now—knowing when the candidate has finished speaking, when to probe deeper, when to move on. But they're terrible at knowing why to do those things.
Our first version would ask "Tell me about a time you faced a technical challenge" and then immediately jump to the next question, regardless of whether the candidate gave a three-sentence answer or a three-minute story. It felt robotic because it was robotic—no human interviewer would blow past a shallow answer without following up.
We had to build a layer that analyzes response depth and triggers follow-ups. Not just keyword matching—actual semantic understanding of whether the candidate addressed the question or danced around it.
This meant combining LLaMA 3.3 70B for conversation flow with TinyBERT for real-time classification. The large model decides what to ask, the small model decides if the answer was substantive enough to move forward. They run in parallel with about 800ms latency between candidate finishing and AI responding.
That 800ms pause? Candidates tell us it makes the conversation feel more natural. Humans don't respond instantly either.
The Bias Problem That Wasn't a Bias Problem
Everyone asked about bias in AI hiring. "How do you prevent discrimination against protected classes?"
Honest answer? We can't. Not completely.
But we can be transparent about where bias enters the system and give recruiters tools to catch it.
Our approach:
Standardized questions - Every candidate gets asked the same core questions in the same order. This eliminates the biggest source of interviewer bias: one person getting softball questions while another gets grilled.
Anonymized analysis - The AI evaluation doesn't see candidate names, photos, or demographic data. It only sees the transcript and voice characteristics relevant to communication (clarity, pace, coherence—not accent or gender).
Bias audit logs - We track which candidates get follow-up questions and why. If the AI is consistently probing deeper with one demographic group, that pattern surfaces in our analytics.
Human override - Recruiters see the full transcript alongside the AI summary. They can—and do—disagree with the AI's assessment.
The dirty secret of AI hiring tools is that removing human bias is impossible. What's possible is making bias visible and consistent. A human interviewer might grill technical candidates on Tuesdays because they're stressed, then lob softballs on Fridays when they're in a good mood. The AI applies the same standards at 2 PM and 2 AM.
That's not unbiased. It's consistently biased, which is actually useful if you know what you're looking for.
What Breaking Things Taught Me
When we started testing the system, the AI asked a great opening question, then froze for 14 seconds before asking it again. The candidate thought the system crashed and hung up.
The bug? Our conversation state management couldn't handle the candidate pausing to think. The silence triggered a "no response detected" error, which triggered a retry, which created a race condition.
Fixed it by adding a confidence threshold—the AI now distinguishes between "finished talking" silence and "still thinking" silence based on speech patterns in the previous 3 seconds. Not perfect, but it dropped the false-positive rate from 18% to 2%.
Here's the lesson I took away: voice AI in high-stakes scenarios requires defensive design at every layer. Unlike a chatbot where someone can retype their message, you can't ask a candidate to "restart the interview" because your error handling failed.
We built in:
Automatic session recovery if connectivity drops
Manual override for recruiters to flag bad transcriptions
A "pause interview" button for candidates (surprisingly popular)
Playback of the actual audio alongside transcripts
The goal isn't perfection. It's resilience when things go wrong, because they will go wrong.
Why This Matters for Other AI Builders
If you're building AI for professional contexts—interviews, legal analysis, medical screening, financial advice—here's what I'd tell you:
Voice is worth the pain. The richness of verbal communication unlocks insights that text can't capture. But only if you're willing to solve the hard problems instead of shipping a minimum viable chatbot.
Domain-specific fine-tuning isn't optional. General-purpose models are amazing and terrible at the same time. They'll handle 90% of your use case brilliantly, then catastrophically fail on the 10% that matters most. Find that 10% early and train specifically for it.
Latency is a feature. We obsessed over response time at first, trying to get under 500ms. Then we realized that instant responses felt uncanny. The sweet spot for conversational AI is 600-1000ms—fast enough to feel responsive, slow enough to feel natural.
Build for the failure modes. Your AI will misunderstand accents, mishear technical terms, and ask nonsensical follow-ups. Design the system so humans can catch these failures gracefully instead of catastrophically.
The Uncomfortable Truth About AI Products
Six months into building Squrrel, I had a realization that almost made me quit: the AI isn't the product. The product is the workflow that the AI enables.
Candidates don't care that we use Wav2Vec 2.0 for transcription or LLaMA 3.3 for conversation. They care that they can interview at midnight without scheduling four emails. Recruiters don't care about our evaluation algorithms. They care that they can review 10 candidates in an hour instead of spending all week on phone screens.
The AI is infrastructure. The value is in removing friction from a broken process.
This realization changed everything. We stopped optimizing for model accuracy and started optimizing for user experience. We added features like letting candidates preview questions before starting, because that reduced anxiety and led to better responses—even though it "broke" the blind evaluation model we'd carefully designed.
Turns out, a slightly worse AI that people actually use beats a perfect AI that sits unused because the UX is terrible.
What's Next
We're expanding our pilots and learning every day. The technology works. The question now is whether we can scale the human side—onboarding, support, training recruiters to trust but verify AI outputs.
I'm also watching the regulatory space closely. The EU AI Act classifies hiring tools as "high-risk AI systems." New York City requires bias audits for automated employment decision tools. This is good—high-stakes AI should be regulated.
But it also means we need to build compliance into the product from day one, not bolt it on later. Audit trails, explainability, human oversight—these aren't nice-to-haves. They're survival requirements.
If you're building AI products in regulated industries, start designing for compliance now. It's way easier than retrofitting later.
2025-12-28 15:59:53
If you’re starting your journey as a web developer, JavaScript is one word you’ll hear everywhere.
But beginners often ask:
In this blog, we’ll answer all these questions in simple language, without jargon, and with real-world clarity.
JavaScript (JS) is a programming language used to make websites interactive and dynamic.
Without JavaScript:
Examples of what JavaScript does:
HTML:
<button>Click Me</button>
CSS:
button {
background-color: blue;
}
JavaScript:
button.addEventListener("click", () => {
alert("Button clicked!");
});
/*
Adds event listener on the button.
Alert is shown in the browser on button click
*/
All three work together, but JavaScript is what makes the page alive.
JavaScript runs in two main environments:
JavaScript in the Browser
Every modern browser (Chrome, Edge, Firefox, Safari) has a JavaScript engine built into it.
That means:
Examples:
This is called client-side JavaScript.
JavaScript outside the Browser (Node.js)
JavaScript can also run on the server using Node.js.
With Node.js, JavaScript can:
Same language, different environment.
Let’s understand what happens when the browser loads JavaScript.
You don’t see this process, but it happens every time a page loads.
What Is a JavaScript Engine?
A JavaScript engine is a program inside the browser that:
Every browser has one:
You don’t need to install anything — it’s built in.
Let’s quickly recap:
If you’re starting JavaScript, this understanding will make everything else easier.
References
MDN Web Docs – DOM Introduction
V8 JavaScript Engine
2025-12-28 15:58:37
Most of us learned algorithms the same way:
For simple cases, that works. But once you hit sorting edge cases, recursion, trees, or ML concepts, things get fuzzy fast.
I built Learn-Algo to fix exactly that problem.
Instead of reading about algorithms, Learn-Algo lets you watch them execute step by step, pause them, rewind them, and experiment with inputs — the same way you’d debug real code.
In this post, I’ll walk through:
No theory overload. No math walls. Just how algorithms actually behave.
Algorithms aren’t static — they’re processes.
When we only read code like this:
for i in range(n):
for j in range(0, n - i - 1):
if arr[j] > arr[j + 1]:
swap(arr[j], arr[j + 1])
we have to mentally simulate:
That mental simulation is the hard part.
Learn-Algo offloads that cognitive load by rendering each step visually:
You stop guessing and start observing.
Let’s take a simple example.
When you open a sorting algorithm in Learn-Algo, you don’t just click “Run”.
You can:
As the algorithm runs, you see:
This instantly answers questions like:
These are things most tutorials say, but rarely show.
The same idea applies beyond classic DSA.
For machine learning concepts like:
Learn-Algo visualizes:
This is especially helpful if you’re coming from a programming background and find ML math intimidating at first.
This walkthrough is for you if:
You don’t need advanced math or deep CS theory to get value — just curiosity.
If algorithms ever felt abstract or “magical”, this is about making them predictable and understandable.
👉 Explore the playground: https://learn-algo.com/