BIPI
BIPI

Reverse Engineering Binaries: Ghidra, x64dbg, and Practical Analysis Techniques

Cybersecurity

Practical techniques for reverse engineering compiled binaries — static decompilation with Ghidra, dynamic debugging with x64dbg, anti-analysis bypass, and identifying cryptographic routines.

By Arjun Raghavan, Security & Systems Lead, BIPI · May 4, 2025 · 16 min read

#reverse-engineering#ghidra#x64dbg#malware-analysis#binary-analysis

Reverse engineering is equal parts patience and pattern recognition. Whether you are analysing a malware sample, a closed-source server binary, or a firmware blob, the workflow is consistent: orient statically, confirm dynamically, document everything.

Initial triage

  • file, strings, and hexdump for format and obvious indicators
  • PE Studio or Detect-It-Easy for packer/protector identification
  • floss for obfuscated string extraction from malware samples
  • pestudio imports tab: which API calls reveal intent (VirtualAlloc, CreateRemoteThread, WSAStartup)

Static analysis with Ghidra

Ghidra auto-analyses binaries on import, producing a decompiler view that is often good enough to read control flow directly. Rename variables and functions immediately as you understand them — context snowballs quickly.

  1. Accept default analysis options; enable aggressive instruction finder for shellcode
  2. Start at entry() → WinMain/DllMain, trace call graph outward
  3. Use 'Find References' on suspicious strings and imports
  4. YARA-style pattern search with Ghidra scripts for crypto constants

Dynamic analysis with x64dbg

Set breakpoints on key Windows APIs before running: CreateProcess, WriteProcessMemory, RegSetValueEx, and InternetConnect. Step through with F7/F8. The arguments register tells you exactly what the binary is doing.

Anti-analysis bypass techniques

  • IsDebuggerPresent patch: NOP the check or manually set PEB.BeingDebugged = 0
  • Timing checks: patch RDTSC-based delays with NOP sleds
  • VM detection: hook CPUID, RDMSR responses via ScyllaHide plugin
  • String obfuscation: set breakpoint after deobfuscation loop, dump result from stack/heap

Identifying cryptographic routines

Custom crypto is a red flag; standard crypto constants in code are a shortcut to understanding data flows. FindCrypt Ghidra plugin searches for AES S-boxes, DES permutations, and RC4 state init loops. XOR-heavy loops with 16 or 32-byte blocks suggest custom crypto worth reversing fully.

Documenting findings

Export annotated Ghidra projects and x64dbg databases. Write a one-paragraph executive summary per malware family covering: infection vector, persistence mechanism, C2 protocol, and data exfiltrated. YARA rules from unique byte sequences complete the deliverable.

high
Average Ghidra decompilation accuracy on x86-64
15-30 min
Time to first network IOC from unknown binary
majority
Malware families using standard crypto (AES/RC4)
Rename one variable and the function becomes readable. Rename five and the whole binary starts to talk.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.