Mastering Reverse Engineering : Re-engineer Your Ethical Hacking Skills

Implement reverse engineering techniques to analyze software, exploit software targets, and defend against security threats like malware and viruses. Key Features Analyze and improvise software and hardware with real-world examples Learn advanced debugging and patching techniques with tools such as IDA Pro, x86dbg, and Radare2. Explore modern security techniques to identify, exploit, and avoid cyber threats Book Description If you want to analyze software in order to exploit its weaknesses and strengthen its defenses, then you should explore reverse engineering. Reverse Engineering is a hackerfriendly tool used to expose security flaws and questionable privacy practices.In this book, you will learn how to analyse software even without having access to its source code or design documents. You will start off by learning the low-level language used to communicate with the computer and then move on to covering reverse engineering techniques. Next, you will explore analysis techniques using real-world tools such as IDA Pro and x86dbg. As you progress through the chapters, you will walk through use cases encountered in reverse engineering, such as encryption and compression, used to obfuscate code, and how to to identify and overcome anti-debugging and anti-analysis tricks. Lastly, you will learn how to analyse other types of files that contain code. By the end of this book, you will have the confidence to perform reverse engineering. What you will learn Learn core reverse engineering Identify and extract malware components Explore the tools used for reverse engineering Run programs under non-native operating systems Understand binary obfuscation techniques Identify and analyze anti-debugging and anti-analysis tricks Who this book is for If you are a security engineer or analyst or a system programmer and want to use reverse engineering to improve your software and hardware, this is the book for you. You will also find this book useful if you are a developer who wants to explore and learn reverse engineering. Having some programming/shell scripting knowledge is an added advantage. Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you Cover......Page 1 Title Page......Page 2 Copyright and Credits......Page 3 Packt Upsell......Page 4 Contributors......Page 5 Table of Contents......Page 7 Preface......Page 13 Reverse engineering......Page 18 Seeking approval......Page 21 Reporting......Page 22 Binary analysis tools......Page 23 Debuggers......Page 25 Malware handling......Page 26 Basic analysis lab setup......Page 27 Our setup......Page 28 Samples......Page 33 Summary......Page 34 Chapter 2: Identification and Extraction of Hidden Components......Page 35 The filesystem......Page 36 Memory......Page 38 The registry system......Page 39 Typical malware behavior......Page 40 Persistence......Page 41 Run keys......Page 42 Load and Run values......Page 43 Startup values......Page 46 The Image File Execution Options key......Page 47 Malware delivery......Page 48 Email......Page 49 Instant messenger......Page 50 The computer network......Page 51 Media storage......Page 52 Exploits and compromised websites......Page 53 Malware file properties......Page 55 Payload – the evil within......Page 56 Tools......Page 57 Autoruns......Page 59 The Process explorer......Page 60 Further reading......Page 63 Chapter 3: The Low-Level Language......Page 64 Bases......Page 65 Converting between bases......Page 66 Binary arithmetic......Page 68 Signed numbers......Page 69 x86......Page 70 Registers......Page 71 Endianness......Page 74 Basic instructions......Page 75 Copying data......Page 76 MOV and LEA......Page 77 Arithmetic operations......Page 78 Addition and subtraction......Page 79 Multiplication and division instructions......Page 80 Other signed operations......Page 81 Bitwise algebra......Page 82 Control flow......Page 84 Stack manipulation......Page 86 Tools – builder and debugger......Page 87 MASM......Page 88 NASM......Page 90 FASM......Page 91 WinDbg......Page 92 Ollydebug......Page 94 Hello World......Page 95 It works!......Page 96 Dissecting the program ......Page 99 Common Windows API libraries......Page 104 Debugging......Page 105 Further reading......Page 114 Chapter 4: Static and Dynamic Reversing......Page 115 Static analysis......Page 116 PEid and TrID......Page 117 MASTIFF......Page 119 PE executables......Page 121 IDA (Interactive Disassembler)......Page 127 Dynamic analysis......Page 128 Memory regions and the mapping of a process......Page 129 Post-execution differences......Page 133 Try it yourself......Page 134 References......Page 143 Analysis environments......Page 144 Virtual machines......Page 145 Windows......Page 146 Information gathering tools......Page 147 Hash identifying......Page 148 Monitoring tools......Page 149 Disassemblers......Page 150 Debuggers......Page 151 Decompilers......Page 152 Network tools......Page 153 Attack tools......Page 154 Software forensic tools......Page 155 Automated dynamic analysis......Page 156 Online service sites......Page 157 Summary......Page 158 Setup......Page 159 Linux executable – hello world......Page 160 dlroW olleH......Page 161 Dynamic analysis......Page 168 Going further with debugging......Page 170 A better debugger......Page 177 Hello World in Radare2......Page 178 What is the password?......Page 184 Network traffic analysis......Page 192 Further reading......Page 198 Hello World......Page 199 Learning about the APIs......Page 200 Keylogger......Page 201 regenum......Page 203 processlist......Page 205 Encrypting and decrypting a file......Page 206 The server......Page 210 What is the password?......Page 212 Static analysis......Page 213 Deadlisting......Page 217 Dynamic analysis with debugging......Page 234 Decompilers......Page 242 Further reading......Page 244 Chapter 8: Sandboxing - Virtualization as a Component for RE......Page 245 Emulation......Page 246 Emulators......Page 247 Linux ARM guest in QEMU......Page 248 MBR debugging with Bochs......Page 250 Further Reading......Page 259 Data assembly on the stack......Page 260 Code assembly......Page 262 Loop codes......Page 264 Simple arithmetic......Page 265 Simple XOR decryption......Page 266 Assembly of data in other memory regions......Page 267 Decrypting with x86dbg......Page 268 Control flow flattening obfuscation......Page 271 Code obfuscation with a metamorphic engine......Page 274 Dynamic library loading......Page 277 Use of PEB information......Page 278 Summary......Page 279 Chapter 10: Packing and Encryption......Page 280 A quick review on how native executables are loaded by the OS......Page 281 Packers or compressors......Page 284 Crypters......Page 286 Obfuscators......Page 288 Protectors......Page 289 SFX Self-extracting archives......Page 290 Debugging though the packer......Page 291 Memory dumping with VirtualBox......Page 305 Extracting the process to a file using Volatility......Page 306 How about an executable in its unpacked state?......Page 309 Other file-types......Page 312 Summary......Page 316 Anti-debugging tricks......Page 317 IsDebuggerPresent......Page 318 Debug flags in the PEB......Page 319 Timing tricks......Page 321 Passing code execution via SEH......Page 322 Causing exceptions......Page 325 Anti-VM tricks......Page 326 Existence of VM files and directories......Page 327 Registry entries made by VMs......Page 328 CPUID results......Page 329 Anti-emulation tricks......Page 330 Anti-dumping tricks......Page 331 Summary......Page 332 Things to prepare......Page 333 Initial static analysis......Page 334 Initial file information......Page 335 Deadlisting......Page 341 Debugging......Page 353 The unknown image......Page 363 Analysis summary......Page 387 Further Reading......Page 389 Analysis of HTML scripts......Page 390 MS Office macro analysis......Page 397 PDF file analysis......Page 401 SWFTools......Page 403 Flare......Page 405 XXXSWF......Page 406 JPEXS SWF decompiler ......Page 407 Summary......Page 411 Further reading......Page 412 Other Books You May Enjoy......Page 413 Index......Page 416 If you want to analyze software in order to exploit its weaknesses and strengthen its defenses, then you should explore reverse engineering. Reverse Engineering is a hackerfriendly tool used to expose security flaws and questionable privacy practices. In this book, you will learn how to analyse software even without having access to its source code or design documents. You will start off by learning the low-level language used to communicate with the computer and then move on to covering reverse engineering techniques. Next, you will explore analysis techniques using real-world tools such as IDA Pro and x86dbg. As you progress through the chapters, you will walk through use cases encountered in reverse engineering, such as encryption and compression, used to obfuscate code, and how to to identify and overcome anti-debugging and anti-analysis tricks. Lastly, you will learn how to analyse other types of files that contain code. By the end of this book, you will have the confidence to perform reverse engineering. -- back cover