Adroit Photo Forensics 2010

Feature Overview

Adroit Photo Forensics is one of the most advanced and most actively developed products in the industry. Adroit Photo Forensics 2010 adds unique new features to help speed-up investigations even more. A quick guide to all the features is available below.

Quick Guide to APF 2010 Features
Fast Case Processing
Automatically generate case details based on selected evidence.
Options to balance between speed and recovery.
GUI is built from ground up to handle digital photo evidence.
Batch processing to analyze multiple cases automatically.
Comprehensive Evidence Support
Supports Encase disk images (single and split)
Supports RAW/DD images (single and split)
Rock Solid Integrity
Generate MD5 and SHA1/SHA256 hashes of photos recovered.
Generate reports with precise block range locations of photos.
Clearly identifies partial photos and form of recovery for photos.
Export hashes of photos recovered into FTK.
Recoveries backed by award-winning and scientifically proven research.
Enhanced Recovery
Full Active Recovery support for NTFS/FAT/HFS/HFS+
Carving Support for unallocated space and slack space between partitions.
Separates photos correctly carved and partially carved.
Enhanced Carving
Validated sequential carving.
LogCarving, recovers photos based on file system logs and validation.
SmartCarving™ recovers fragmented photos automatically.
GuidedCarving™ recovers fragmented photos manually.
Photo Format Support
JPEG and camera RAW formats.
PNG, GIF (active), and BMP.
DNG and TIFF support
Photo Details Support
Complete File System Information of photos
Complete Header Data information.
Complete Metadata info (EXIF/IPTC).
Embedded thumbnail identification.
Block data linking for JPEGs.
Photo Grouping/Filtering
Photo specific grouping based on camera, EXIF Date stamps etc.
Standard grouping based on File name, date, size etc.
Enhanced Timeline grouping based on file modification/creation/access or EXIF dates.
Content Based SmartFiltering™
Different Quality/Speed settings for Explicit Image Detection
Child Explicit Image Detection.
Face Detection.
Thumbnail - Image mismatch detection.
Duplicate photo using hash detection.


EnCase EnScript "Memory Forensic Toolkit" Version 1.81


I've released three modules: PsEntropyPEB, PsEntropyVAD and VadDump for XP 32bit image.


PsEntropyPEB EnScript calculates Entorpy value of runnning processes based on PEB (Process Environment Block) information.


Unfamiliar 5 executables has similar codes. Basically, the result is free of the influence of packing because the calculation is applied to unpacked code inside RAM.
PsEntropyVAD finds code-injected processes by checking flags of VAD (Virtual Address Descriptor) entries.


We can make sure Metasploit Meterpreter injected malicious code to some processes.
VadDump EnScript exports process memories by traversing VAD trees. For example, when one process is judged as injected process after executing PsEntropyVAD, you can use VadDump to export the suspicious memory pages.


You can specify one process or all processes to export, and if you check "Injected Memory Pages Only", the script exports only suspicious pages.


If you also check "Debug Mode", the exported pages are displayed on Console Tab.


One of exported pages has malicious code.

This is another example.


Is the result indicating the process is injected?
Eventually, you should analyze the code ;-)
I've fixed a bug of VadSearch EnScript. The fixed version adds search-hit keywords to bookmark.


Memory forensic tool

MANDIANT Memoryze is free memory forensic software that helps incident responders find evil in live memory. Memoryze can acquire and/or analyze memory images, and on live systems can include the paging file in its analysis.

MANDIANT Memoryze features:
  • image the full range of system memory (not reliant on API calls).
  • image a process’ entire address space to disk. This includes a process’ loaded DLLs, EXEs, heaps, and stacks.
  • image a specified driver or all drivers loaded in memory to disk.
  • enumerate all running processes (including those hidden by rootkits). For each process, Memoryze can:
    • report all open handles in a process (for example, all files, registry keys, etc.).
    • list the virtual address space of a given process including:
      • displaying all loaded DLLs.
      • displaying all allocated portions of the heap and execution stack.
    • list all network sockets that the process has open, including any hidden by rootkits.
    • specify the tunctions imported by the EXE and DLLs.
    • specify the functions exported by the EXE and DLLs.
    • hash the EXE and DLLs in the process address space> (MD5, SHA1, SHA256.  This is disk based.)
    • verify the digital signatures of the EXE and DLLs. (This is disk based.)
    • output all strings in memory on a per process basis.
  • identify all drivers loaded in memory, including those hidden by rootkits. For each driver, Memoryze can:
    • specify the functions the driver imports.
    • specify the functions the driver exports.
    • hash the driver. (MD5, SHA1, SHA256. this is disk based.)
    • verify the digital signature of the driver (This is disk based.)
    • output all strings in memory on a per driver base.
  • report device and driver layering, which can be used to intercept network packets, keystrokes and file activity.
  • identify all loaded kernel modules by walking a linked list.
  • identify hooks (often used by rootkits) in the System Call Table, the Interrupt Descriptor Tables (IDTs), and driver function tables (IRP tables).

    MANDIANT Memoryze can perform all these functions on live system memory or memory image files – whether they were acquired by Memoryze or other memory acquisition tools. 
    Memoryze supports analysis of the following operating systems:
    • Windows 2000 Service Pack 4
    • Windows XP Service Pack 2 and and Service Pack 3 (32-bit)
    • Windows 2003 Service Pack 2 (32-bit)
    • Windows Vista Service Pack 1 and Service Pack 2 (32-bit)
    • Windows 2003 Service Pack 2 (64-bit)
    In order to visualize Memoryze's output, please download Audit Viewer.  Audit Viewer is an open source tool that allows users to examine hte results of Memoryze's analysis.  Audit Viewer allows the incident responder or forensic analyst to quickly view complex XML output in an easily readable format.  Using familiar grouping of data and search capabilities, Audit Viewer makes memory analysis quicker and more intuitive.
    Check out the ways you can use Memoryze. 
    And, if you like Memoryze's standalone capabilities, check out MANDIANT Intelligent Response. It's our enterprise-grade incident response accelerator. MIR has all the memory forensics of Memoryze, plus a lot more... making enterprise live response faster and easier, especially for teams of responders. Imagine Memoryze doing deep memory forensics on thousands of machines at a time, then having the results easily searchable to find where evil is hiding across your enterprise. Then add disk analysis and live response. That's Intelligent Response.


    Gatherer Transaction Log Files - a Windows Search artefact

    A recurring theme in many examinations is the prevalence of evidence in unallocated clusters. Reinstallation of the OS is often to blame and a recent case where XP was installed on a drive where the previous OS was Vista further complicated matters. All relevant data had been created during Vista's reign and the challenge was to determine what files and folders existed under this OS. The Encase Recover Folders feature assisted to an extent as did Digital Detective's Hstex 3. Loading the output of Hstex 3 into NetAnalysis allowed me to identify the download of a number of suspect files and some local file access to files within the Downloads folder.

    The next step was to carry out a keyword search utilising the suspect file names as keywords. This is always a good technique and results in the identification of useful evidence in a variety of artefacts (e.g. index.dats, link files, registry entries, NTFS file system artefacts et al) but because in this case every thing was unallocated identifying all the artefacts was a little tricky. A considerable number of the search hits were clearly within some structured data but the data was not an artefact I was familiar with.

    I have highlighted Record Entry Headers to draw attention to the structured nature of the data. This screen shot is of test data where the file names/path are stored as unicode as opposed to ASCII in the case I was investigating.

    A bit of googling led me to page 42 of Forensic Implications of Windows Vista - Barrie Stewart which identified the structured data I had located as being part of Gatherer Transaction Log files created by the search indexer process of Windows Search. These files have a filename in the format SystemIndex.NtfyX.gthr where the X is replaced by a decimal number and on a live Vista system can be found at the path

    These files have the words Microsoft Search Gatherer Transaction Log. Format Version 4.9 as a file header. The files are a transaction log of entries committed to the Windows search database indexing queues. The SearchIndexer process monitors the USN Change Journal which is part of the NTFS file system used to track changes to a volume. When a change is detected (by the creation of a new file for example) the SearchIndexer is notified and the file (providing it is in an indexable location - mainly User folders) is added to the queue to be indexed. The USN Change Journal is also something that may contain evidentially useful information and I will look at it in more depth in the next blog post.
    Sometimes artefacts are only of academic interest but it was fairly apparent that this data could have some evidential value. Each file or folder has a record entry; parts of which had been deconstructed by Stewart. I was able to identify two additional pieces of information within each record - the length of the Filename block and a value that is possibly a sequence or index number or used to denote priority. I also observed some variations in some parts of the record that had been constant in Stewart's test data.
    • Record Header 0x4D444D44 [4 bytes]
    • Unknown variable data [12 bytes]
    • FILETIME Entry [8 bytes]
    • FILETIME Entry [8 bytes] or a value of 0x[0100]00000000000000
    • Unknown variable data [12 bytes]
    • Length of file path following plus 1 byte (or plus 2 bytes if file path stored as unicode) [4 bytes] stored as 32 bit integer
    • Name and fullpath of file/folder (ASCII or Unicode -version dependant) [variable length]
    • 0x000000000000000000FFFFFFFF [13 bytes]
    • FILETIME Entry [8 bytes] or a value of 0x[0100]00000000000000
    • 0xFFFFFFFF [4 bytes]
    • FILETIME Entry [8 bytes] or a value of 0x[0100]00000000000000
    • Unknown variable data [4 bytes]
    • Sequence or index number? [1 byte] stored as 8 bit integer
    • Unknown variable data [15 bytes]
    • FILETIME Entry [8 bytes] or a value of 0x[0100]00000000000000
    • FILETIME Entry [8 bytes] or a value of 0x[0100]00000000000000
    • Unknown variable data [20 bytes]
    Microsoft do not seem to have publicly documented the record structure. To establish how useful this data can be I came to the conclusion that I needed to recover all of these records from unallocated. I needed an enscript and Oliver Smith over at Cy4or kindly wrote one for me. I wanted the enscript to parse out the file and path information, sequence or index number, the six time stamps and a hex representation of each unknown range of data into a spreadsheet. The script searches for and parses individual records (from the live systemindex file and unallocated) as opposed to entire files. I was astonished at just how much information the script parsed out - email me if you want a copy. Setting the spreadsheet to use a fixed width font (Courier New) lines up the extracted hex very well should anyone want to reverse engineer these records further. As it stands the file paths and timestamps can provide some useful evidential information, particularly when the recovered records have been recovered from unallocated clusters and relate to a file system older than the current one.
    Obviously once you have run this enscript or manually examined the records the first question that arises is what are the timestamps. Establishing this has not been as easy as it could be and hopefully a little bit of crowd sourcing will sort this out for all of us. Post a comment if you can help in this regard. One approach is to use the hex 64 bit filetime value as a keyword and see where you get hits. Hits in another timestamp indicates that the timestamp is the same down to the nanosecond. Carrying out this process will result in hits in OS system files and fragments of them. I have found on the limited test data set I have used that Timestamp 3 matched the File Modified (File Altered) date within MFT for the file concerned and the timestamp for the same file in the USN Change Journal. The timestamp in the USN Change Journal record is the absolute system time that the change journal event was logged (1). It is worth reminding readers who are Encase users that Encase uses different terminology for the time stamps within the MFT - file modified is referred to as Last Written. I think it likely that timestamps 1 and 2 are linked to the indexing function (e.g. time submitted for indexing) given the journalling nature of the file but can not either prove this by testing or confirm this within Microsoft documentation. I can say that in testing sorting on Timestamp 1 gave a clear timeline of the file system activity I had provoked within User accessible folders.

    Example CSV output of Enscript (click to enlarge)



    網路分享器廠商們請注意,假如您的設備剛好是用 Web 頁面管理,採用 DD-WRT 或 OpenWRT Linux-based 的韌體,那麼快趁這僅存的幾天想好對策吧。因為 Seismic 的 Craig Heffner 宣稱他利用千年傳統.全新感受的入侵方法 - DNS rebinding,成功地入侵了許多台分享器,並且預計在下個星期於 Black Hat 2010 中發表這項研究成果,屆時地球上將會有數以百萬計的分享器生活在隨時被入侵的陰影之中。

    目前 Heffner 嘗試攻擊了 30 種不同型號的分享器,一半以上都應聲而倒,裡頭還包括數款常見的分享器,如上圖的 Linksys WRT54G。雖然一般而言,換掉預設的密碼就可以預防大部份的駭客入侵。然而 Heffner 相信根本的解決之道還是得由廠商盡快推出修正程式,才能讓普羅大眾免受這場災難。延伸閱讀經測試過的分享器列表,各位不妨看看家裡使用的那台是否也在名單 之中。

    Read - 清單
    Read - Forbes
    Read - Ars Technica
    Read - Black Hat 2010


    LNK新型態隨身碟病毒 - 分析解決方案

    此病毒不 再使用autorun.inf自動執行的方式感染電腦
    而是製作一個.lnk類型檔案指向病毒檔案 利用該弱點在系統中執行


    1. Ivanlef0u's Blog CVE-2010-2568 shorcut Lnk + PoC (Google translated to English)
    2. Exploitdb Microsoft Windows Automatic LNK Shortcut File Code Execution (PoC by Ivanf0u)
    3. Microsoft Security Advisory (2286198) Vulnerability in Windows Shell Could Allow Remote Code Execution
    4. Brian Krebs Experts Warn of New Windows Shortcut Flaw
    5. InReverse  About TmpHider/Stuxnet #1 by swirl
    6. Wilders Security Forums - Rootkit.TmpHider
    7. Microsoft Malware Protection Center - The Stuxnet Sting
    8. Microsoft Malware Protection Center - WinNT/Stuxnet.A
    9. Threatexpert - Win32/Stuxnet.A
    10. ESET (Windows) Shellshocked, Or Why Win32/Stuxnet Sux… by David Harley (with special thanks to Juraj Malcho, Aleksander Matrosov and their colleagues)
    11. Aleksander Matrosov "Rootkit.TmpHider is signed with signature of Realtek Corp"" /via @_MDL_ 
    12. Sophos Windows shortcut vulnerability with rootkit - detailed video demo 
    13. Mitigating .LNK Exploitation With Ariad — Didier Stevens 
    14. Windows zero-day attack works on all Windows systems by Chester Wisniewski



    016169ebebf1cec2aad6c7f0d0ee9026  received on 2010.07.16 11:55:58 (UTC)
    Result: 25/41 (60.98%)
    a-squared     2010.07.16   
    AhnLab-V3     2010.07.16.00     2010.07.15   
    AntiVir     2010.07.16     
    Avast     4.8.1351.0     2010.07.16     
    Avast5     5.0.332.0     2010.07.16   
    AVG     2010.07.16   
    BitDefender     7.2     2010.07.16   
    Comodo     5446     2010.07.16     
    DrWeb     2010.07.16
    F-Secure     9.0.15370.0     2010.07.16   
    GData     21     2010.07.16     
    Ikarus     T3.     2010.07.16   
    Kaspersky     2010.07.16   
    McAfee     5.400.0.1158     2010.07.16
    McAfee-GW-Edition     2010.1     2010.07.16   
    Microsoft     1.6004     2010.07.16   
    NOD32     5283     2010.07.16   
    nProtect     2010-07-16.01     2010.07.16     
    PCTools     2010.07.16     
    Prevx     3.0     2010.07.16     
    Medium Risk Malware
    Sophos     4.55.0     2010.07.16     
    Sunbelt     6591     2010.07.16   
    Symantec     20101.1.1.7     2010.07.16   
    VBA32     2010.07.16   
    VirusBuster     2010.07.16     
    Additional information
    File size: 517632 bytes
    MD5   : 74ddc49a7c121a61b8d06c03f92d0c13



    微軟發布新的0-day弱點 USB病毒新型態攻擊

    微軟發布新的0-day弱點 USB病毒新型態攻擊

    而是製作一個.lnk類型檔案指向病毒檔案 利用該弱點在系統中執行
    另外病毒也會產生兩個具有Rootkit性質的惡意檔案 偵測為RTKT_STUXNET.A


    美印組成網路聯合部隊 對抗中共網軍

    美印組成網路聯合部隊 對抗中共網軍


      2010年1月19至21日,美國國防部長蓋茲(Robert Gates)訪問印度,除了與印度國防部官員進行會晤外,面對日益險峻的網路安全威脅,蓋茲提出美國將協助印度這個新結交的盟友建立資訊戰部隊,以共同對 抗中共網路部隊的威脅。這是繼2009年美印聯合軍演之後,美、印兩國再度攜手合作,此舉也為美、印、中複雜的三角關係再度埋下伏筆。


         「網軍」這個名詞,首見於1999年的中共解放軍報上。所謂的網軍,泛指攻擊它國電腦網路與保護自身電腦網路系統免於攻擊的網路部隊。雖然中共自始自終不 願承認有這隻網路部隊的存在,但是令人感到弔詭的是,各種網路戰、信息戰(中共稱網路作戰為信息戰)的相關研究文章,卻充斥於解放軍的各級出版品中,由此 不難看出解放軍對於資訊戰已經能充分了解與運用。

          2001年4月1日,在南中國海發生「中」美軍機擦撞事件之後,美國與中國大陸駭客開始大規模密集交鋒。事發之後,中國大陸最大的駭客組織「紅客聯盟」 號召大陸網民對美國各大網站進行網路攻擊。在這波網路攻擊中,高達800個美國政府及企業網站因而「受駭」,網頁不是變成中共五星旗,就是殉職飛行員王偉 的遺照,這也讓美國首次領教到中國大陸駭客的厲害。

         有了這次交手經驗後,中共對美國的資訊攻擊與竊密活動就不斷增加,美國軍方和政府部門的電腦網路受到了中共駭客的持續性攻擊。華盛頓的中共軍事專家詹姆士 •穆文濃(Jams Mulvenon)說:「和美國不同,中共基本上把網路視為一種國力的工具。」利用網路達到戰略與政治目的,被中共方面視為一種理所當然的手段。

         另一方面,美國也以其人之道還致其人之身。美國國防部、中央情報局以及美國國家安全局等情報單位,也時常入侵中共軍事和工業領域的電腦網路,以獲取相關 機密情報。一般預料,一旦美國與中共發生軍事衝突,美國及其北大西洋公約盟友更會嘗試破壞中國大陸的電腦網路,透過網路攻擊癱瘓中國大陸的國家與社會運 作。

        2009年,根據「美中經濟暨安全審查委員會」(US-China Economic and Security Review Commission, USCC)提交的《2008年對國會報告》中指出(註一),中國大陸目前約有250個駭客組織,這些組織受到官方的嚴密監控,甚至可能受到官方的鼓勵,入 侵其他國家網路。報告引述美國戰略司令部(U.S. Strategic Command)主管全球網路行動的麥克阿隆上校(Colonel Gary McAlum)的話:「中共日漸意識到以網路作為一種新型態戰爭工具的重要性。」因此中共將軍事事務革新的重點聚焦在資訊戰發展上,藉由不斷地加強資訊戰 相關訓練,以取得在任何時間、任何地點的資訊戰優勢。

          雖然中國大陸一再拒絕承認「網軍」的存在,不過美國有線電視新聞網(CNN)曾報導指出,部份入侵美國國防部的中國大陸民間駭客,其實幕後獲得官方的資金 援助,其目的是為了獲取機密情報與試探美軍的資訊戰防禦能力。

          由於電腦與網路已被廣泛運用在政治、社會、經濟、文化與國防等各個層面,所以圍繞網路所進行的攻擊活動也隨之應運而生。這類網路攻擊活動所花費的成本比傳 統恐怖活動低廉,襲擊者也不必犧牲自己的生命,但卻可以造成敵國在軍事、經濟和社會等各方面的大規模癱瘓與混亂,是一種以極小成本獲取極大收益的攻擊行 動。因此,自從911恐怖攻擊事件之後,美國的國家安全單位除了要擔憂恐怖襲擊再臨本土外,更要擔憂敵對勢力透過網路對美國發動恐怖攻擊。

        由於美國網路系統不時遭到中共駭客入侵,所以美國自然而然地視中共為主要網路戰假想敵。2008年3月,美國舉辦第二次「網路風暴」演習(Cyber Storm II),模擬了外國網軍、恐怖組織和「陰謀分子」襲擊美國的通訊、化工與運輸部門的情況。儘管這次演習主辦單位不願公開承認,事實上這次演習模擬外國網路 攻擊部份,就是模擬中共駭客對美國的網路攻擊。由此可知,美國已將來自於中國大陸的網路攻擊視為國家安全的主要威脅。

    美 印網軍聯姻對抗中共網軍

         2010年1月初,美國媒體引述美國聯邦調查局(FBI)的一份機密報告指出,中國大陸已經組建一支超過18萬人的網路軍隊,其中3萬為網路特工、15萬 為民間駭客,目標是在2020年建立全球第一支「資訊化武裝的部隊」。

        2010年1月20日,美國國防部長蓋茲訪問印度時表示,美國準備扶持印度建立網路部隊,一同對抗數量日益龐大的中共網路部隊。這是繼小布希政府時期簽署 民用核能合作、2009年舉辦聯合軍演之後,美國與印度再度攜手合作。美、印之所以攜手合作,出於下列原因:

         第一,印度其實也是網路攻擊的受害者。根據印度媒體《今日印度》在2009年12月的報導指出,印度總理辦公室30多名政要的電腦在2009年12月遭到 來自於中國大陸的駭客入侵,並有部份資料洩漏。《今日印度》預測,這是中共駭客試圖在哥本哈根氣候會議期間,獲取一些關於印度氣候議題立場的情報。

          除了中國大陸之外,印度的主要宿敵巴基斯坦,也正在加速其資訊戰能力發展。因此,印度國內輿論開始呼籲必須建立網路部隊,預防敵國的網路攻擊行動。 2008年時,印度軍方曾提出建立萬人網路部隊的構想,目前已在陸軍總部下設立網路安全部門,並在所有軍區和重要軍事部門建立網路安全分部。印度還對軍校 學員進行「駭客技術」培訓,課程集中在獲取情報和反網路偵察上。
        第二,冷戰時期,由於印度奉行「不結盟外交政策」,更因為印度在1970年代與蘇聯簽訂《印蘇友好條約,使得美印關係相形疏遠。1998年5月,印度不顧 美國反對進行一連串核試爆,迫使柯林頓政府對印度採取相關制裁措施,美印關係陷入空前冰點。

          雖然中國大陸與印度同為崛起的大國,兩國在人口數量、經濟發展上呈現諸多相似之處,但是印度為民主化國家,同時與中國大陸有著戰略利益衝突與歷史宿怨,所 以美國自小布希政府以來,對印度的外交活動開始轉趨熱烈,試圖拉攏印度以圍堵中共勢力的擴張。另一方面,就新德里的觀點來看,與美國密切往來,除了可以制 衡中國大陸崛起之外,更可以聯手打擊印度周遭地區猖獗的恐怖主義,何樂而不為呢。

          美印關係由冷轉熱,也反映在資訊戰方面上。美國和印度官員都認為,即使做最好的打算,中共也會是網路的麻煩製造者,而最壞的打算則是中共有可能成為兩國的 網路敵人,所以美國官員希望,美印兩國能在資訊安全問題方面加強聯繫。

         最後,近年來,印度資訊產業的發達引發了國際社會的密切關注。由於印度貧窮人口眾多,所以許多人認為印度是落後的第三世界國家。事實上,印度自獨立以來, 相當重視高科技軟體人才的培養,印度各大學每年為軟體業界提供數以千計的優秀軟體人才,光是在資訊科技(IT)領域,印度的技術研發人員超過320萬人。

         由於IT產業需要有專業人力、不需要原料,且不會造成環境傷害,可提供年輕人高品質的就業機會,而軟件的外銷又將為印度帶來可觀的外匯收入,因此印度政府 在1980年代中期鼓勵發展IT工業。千禧年之前,歐美先進國家的資訊系統為了因應千禧年(Y2K)危機,需要大量軟體人才來處理程式修改或系統重建,印 度IT產業因緣際會,正好爭取到這絕佳發展外銷軟件市場時機。

        1980年,印度IT業產值只有3000萬美元,到了2009年躍升達710億元,其中三分之二為出口額。IT產業難怪被全民視為印度解決貧窮、邁向新世 紀的最佳良方。由於資訊產業的猛烈發展,印度的經濟也獲得加速度成長,印度政府先後成立七個國家級軟體園區。過去十多年來,印度成為全球的資訊產業發展重 鎮,歐美跨國企業都在印度設有研發基地,西方國家將軟體業務外包給印度成為一種無可避免的趨勢,全球前五百百大企業中,超過半數將軟體開發業務交由印度公 司去執行。

         由此可知,印度優異的軟體研發能力,將有助於兩國發展資訊戰部隊。同時,由於網路搜尋引擎巨人谷歌(Google)將與美國國家安全局聯姻,加上美、印將 組成網路聯合部隊,未來美、中、印在網路這個虛擬空間的爭奪,將不比實體空間來得平和。




         面對日益升高的網路威脅,美國採取攻守兼備的應對措施。在攻擊方面,美軍在2007年2月成立「網路戰聯合功能構成司令部」( Joint Functional Component Command Network Warfare, JFCCNW),其主要職責是對敵人發動網路攻擊。尤其在戰時快速入侵敵方的電腦網路系統,藉此癱瘓敵軍的指揮網路和仰賴電腦維持運行的武器系統。

         在防守方面,美軍「全球網路聯合部隊」(Joint Task Force-Global Network Operations, JTF-GNO)於2002年10月成立,其前身為1998年成立的「全球電腦網路防衛部隊」(Joint Task Force-Computer Network Defense, JTF-CND)。全球網路聯合部隊的主要職責為保護美國軍方在本土與全球範圍內的網路系統安全,每日必需應付高達數十萬起試圖入侵美軍網路系統的攻擊行 動。

         2009年6月,美國國防部將這兩隻矛與盾部隊加以整合,宣布成立網路戰司令部。美國戰略司令部司令希爾頓將軍(General. Kevin P. Chilton)表示,戰略司令部正在徵召2000至4000名士兵,建立一支「網路特種部隊」。這支特種部隊不僅要負責網路防禦的任務,還將對它國的網 路與電子系統進行秘密攻擊。

          儘管美國從未正式公佈網路部隊的人數,但根據長期研究美軍資訊戰發展的軍事專家哈丁(Joel Harding)說:「目前美軍約有三千至五千名資訊戰專家,至少五萬名士兵涉足相關作戰領域。」如果加上原有的電子戰人員編制,美軍在資訊戰人數編制上 應該約在88000人,相當於七個101空降師的編制規模。


         為颱風戰機與英國核子潛艦製造引擎的勞斯萊斯公司,其電腦系統早在2007年就遭到入侵。因此,英國政府通信總部在2008年宣佈將成立新的網路安全作業 中心,負責協調英國國防部、內政部、軍情五處、軍情六處的相關任務,以確保英國的網路安全。

          不過中共駭客在2010年4月成功竊取洛克希德.馬丁(Lockheed Martin)公司的聯合打擊戰機(Joint Strike Fighter)設計計畫與電子系統之後,令英國覺得這樣的防護措施似乎還不夠。所以英國政府主管安全事務的官員表示,將成立新的網際安全作戰中心 (Cyber Security Operations Centre),並招募年輕的駭客做為核心戰力,建立一支擁有反擊能力的強大網路部隊。


         儘管全球大部分地區正接受數位化浪潮洗禮,北韓卻幾乎完全與網路斷絕,只有鋪設部份區域網路,能與外界溝通的全球資訊網卻被徹底隔絕,成為名符其實的網路 黑洞國家。但是北韓卻有網路部隊的存在。

         近20年來,平壤自動化大學與金日成軍事綜合大學每年向北韓人民軍提供100名高級電腦人才,他們集中接受電腦、情報傳送體系、暗號開發、駭客技術等專業 技能的學習和訓練。北韓駭客部隊直接由北韓人民軍總參謀部指揮自動化局和人民武力部偵察局所領導,主要任務包括蒐集有關朝鮮半島開戰時,美國實施軍事增援 計劃的情報,以及為擾亂美國軍事網路和C4ISR系統蒐集資料。目前北韓駭客部隊維持在500人左右,年度預算700多萬美元。

         為求跟得上瞬息萬變的網路世界,這隻網路駭客部隊必須不斷進行人員淘汰、更新電腦知識。美國在對資訊戰的威脅評比等級中,將北韓列為第8位。2009年 11月,關於朝鮮半島作戰基本框架和概念的《5027作戰計劃》的部分資料遭到竊取。南韓軍方推測,這可能是北韓駭客部隊所為。


        雖然南韓為亞洲崛起的消費性電子產業與資訊大國,南韓政府部門由於長期缺乏資金與專家,使得國家網路安全相對薄弱。南韓600多個政府部門機構中,平均每 一個政府機構只有0.7名資訊安全專家負責維護網路安全。經歷了近來一連串的北韓駭客網路襲擊之後,南韓當局毅然而然地決定成立「資訊安全司令部」。新設 的資訊安全司令部將設在國防情報本部麾下,擁有200名電腦專家,由少將級軍官所指揮。預計在2010年初成立,同年7月起正式運作,除了維護南韓國家的 網路資訊安全外,如有必要,還將具備擾亂其它國家網路系統的攻擊能力。


         面對可能存在的網路攻擊潛在威脅,日本政府決定,將著手制定網路攻擊防範對策。 日本政府將著手製定網路攻擊防範對策,並已要求各相關機構做好應對攻擊的準備,還將跟美國等其他國家在網路安全領域上加強合作。日本防衛省已經建立了一支 約5000人的網路戰部隊,主要任務是進行反駭客攻擊,同時研發可以破壞其他國家網路系統的網路武器,必要時可對敵方重要網路實施「癱瘓戰」。

    結 語

          美國著名軍事預測學者詹姆斯‧亞當斯(James Adams)在其著作《下一場世界戰爭》中曾預言:在未來的戰爭中,電腦本身就是一種武器,前線無處不在,奪取戰場控制權將不是炮彈和子彈,而是電腦網 路。如今,世界各大國相繼發展網路戰力,意味著爭奪網路空間的控制權,將一如過去爭奪制海、制空權一樣重要,一場看不到煙硝的戰爭正在如火如荼地進行中。

    註一:美國國會於2000年 10月通過《2001年佛洛德‧斯彭斯國防授權法案》,(Floyd D. Spence National Defense Authorization Act),根據這項法案成立「美中經濟暨安全審查委員會」,目的為監督與和調查美國和中國大陸進行雙邊貿易和經濟關係時,對美國國家安全可能產生的影響。 根據這項國防授權法案規定,該委員會必需針對美國與中國大陸之間交往的八大領域進行監督與研究,並每年向國會提交年度報告。


    隨即祭起「我的電腦」,打開我的工作盤F盤。裡面可是公司十多年的代碼,還有自己東搞西搞十年的東西呢。可是馬上我就呆了,F盤空空如也。比新 裝的電腦還要乾淨:

    老天開始耍我了,我滿滿的裝了快60G的數據啊。一下就給我清空了!第一時間想到的是病毒!回想之前也試過數據丟失,有幾個文件夾無故丟失,D 盤根目錄下的文件也全丟了。由於系統安裝了免費的Avast!殺毒軟件,以前安裝NOD32並沒出現這樣的問題。覺得於是更加肯定了病毒一說!

    回想昨天下班回家後開機,進系統之前刷刷刷閃了幾屏。一時覺得是病毒刪除或格式化了我的F盤。再看了一下日期剛好7月8號,昨天數據丟失就是7 月7號。這應該是病毒了吧?上回D盤文件丟失應該是6月6號吧(我可忘了,當時壓根就不知道)?


    這回確定是病毒格式化了我的F盤。再次用FinalData進行完全掃瞄,提示需要好幾個小時。還好看提示可以找到我的文件。沒辦法,只能耐心 的等待。只可惜的是耐心的等待未必會有結果,等了兩個小時之後電腦自動關機。出乎我的意料啊!再試了一次,還是這結果。

    這次用FinalData掃瞄了半小時就手動停了,查看了一下找到的文件。有部分文件已經找出來,隨意恢復了兩個RAR文件解壓也沒有問題。看 到恢複數據是沒有問題。只是FinalData並沒有找到目錄結構,而且不能完全掃瞄。恢復起來難度大,恢復的文件也沒有什麼意義。誰會記得所有文件該放 哪個目錄哇?

    以前經常用這軟件進行數據恢復,這回用不了。只能使用其他數據恢復軟件。找了一下,PE時有個R-Studio的恢復軟件。打開掃瞄了一下F盤 也沒有找到任何文件(我並沒有進行Scan,只是雙擊F盤的分區掃瞄了一下)。突然想起最新版本的DiskGenius有數據恢復的功能,只能試一下。



    真的幸運,至少公司的代碼找回了。自己的東西丟了也就丟了,公司的東西丟了可就掛了。公司最新的代碼可全在我這啊!至於自己的數據只能再想辦 法,至少目前來看病毒只是破壞了FAT,數據並沒有損壞還是有希望找回來的。


    斷斷續續搞了半天也沒有結果,只能詳細看一下DiskGenius搜索出來的東西。DiskGenius有一個名為「丟失的文件」的文件夾,裡 面裝著N多的文件夾(估計有幾萬個吧)。隨意看了幾個都是些沒有的垃圾,現在沒辦法之下只能查看這裡所有的文件夾了。


    說一下在這裡找文件的技巧吧,看到這裡的文件夾都是以「丟失的文件 {數字}」命名的吧?其實這裡的數據是該文件夾在硬盤上的扇區數,所以如果你要找的文件是很早期的,那先看看扇區數小的文件夾。像我的數據就在41和52 扇區,越後期的數據扇區數越大。這可是我查看所有文件夾得出來的技巧啊!
    最後看了一下恢復出來的數據大小有51G多,F盤總共61G容量。除去原有的2G可用空間,還有一個2.4G的文件和安裝的Eclipse等沒 有恢復出來的垃圾數據。這次恢復幾乎100%恢復成功!
    恢覆文件之後進系統,用殺毒軟件進行全盤掃瞄。竟然沒有找到一個病毒,以為是Avast!殺毒軟件的問題。換NOD32再殺,結果一樣,並沒有 找到一個病毒。
    到這就奇怪了,既然不是病毒。那是誰把F盤格式化了呢?經過一番排查,終於找到兇手。留意第二張圖片,就是FinalData那張,裡面有個 bootex.log文件。
    在這個文件可以看到,是XP的CHKDSK程序檢查F盤。發現所有索引都有問題,所以把所有文件索引都消除掉了。於是就造成了這次數據丟失。以 前從來 沒有出現過這樣的問題,唉。真是M$啊,他不是吹說NTFS格式多穩定多可靠嗎?


    Inside the Native API

    Native API Architecture
    The Windows NT Native API serves one purpose: as a means for calling operating system services located in kernel mode in a controlled manner. Kernel mode is where the core of NT executes, and it's in kernel mode that components have direct access to hardware and services that perform management of the computer's resources including memory, devices and processes. Thus, whenever a program executing in user mode wants to perform I/O, allocate or deallocate virtual memory, start a thread or process, or interact with global resources, it must call upon one or more services that live in kernel mode.

    The Native API is equivalent to the system call interface on traditional monolithic operating systems such as most UNIXes. On most UNIXes, however, the system call interface is well documented and is generally available for use by standard applications. For example, the read() call for reading data from a file, socket, or input device in most flavors of UNIX is a system call that is handled by code in kernel mode. In Windows NT the Native API, its system call interface, is hidden from programmers behind higher level APIs such as Win32, OS/2, POSIX or DOS/Win16. The reason behind this is NT's architecture.
    NT is a "modified microkernel" architecture. Instead of supporting one basic operating system API, NT implements several. It does this efficiently by implementing operating environment subsystems in user mode that export particular APIs to client programs. The "national language" API of NT is Win32, and the Win32 architecture demonstrates this concept. The Win32 operating environment subsystem is divided among a server process, CSRSS.EXE (Client/Server Runtime SubSystem), and client-side DLLs that are linked with programs that use the Win32 API. The core of the Win32 API is divided into three categories: windowing and messaging, drawing, and base services. Windows and messaging APIs include CreateWindow() and SendMessage(), and are exported to Win32 programs via the USER32.DLL library. BitBlt() and LineTo() are Win32 drawing functions and are provided in GDI32.DLL. Finally, base services include all Win32 I/O, process and thread, memory management, and synchronization APIs, and KERNEL32.DLL is the library that exports them.
    When a Win32 program calls a Win32 API control is transferred within its address space into one of Win32's client-side DLLs. The DLL can execute one or more of the following options:
    Ø          return immediately to the caller
    Ø          send a message to the Win32 server to request help
    Ø          invoke Native APIs to carry out the function
    The first option of returning to the caller is rarely possible and is only possible when the DLL can service the function without the help of operating system services. An example of this is GetCurrentProcess(). This API simply returns a handle to the current process that is cached in KERNEL32 when the process is started.
    The second option is also rarely required. A client-side DLL only needs to send messages to the Win32 server when the server must participate with, and be aware of, the function's execution. The Win32 server creates a Win32 execution environment for its clients that involves maintaining some state associated with its client processes. Thus, the CreateProcess() API, exported by KERNEL32, requires an interaction with the Win32 server. The server in this case prepares a new process for execution by mapping in an executable image, creating a command-line argument structure, and so on. The Win32 server calls Native API functions to create the actual process image and prepare its address map.
    The final option is the most frequently exercised. Let's talk about USER32 and GDI32 APIs first, before talking about KERNEL32's use of Native APIs. In versions of NT prior to 4.0, windowing and drawing functions were located in the Win32 server (CSRSS.EXE). This meant that whenever an application used these function a message would be sent to the server. In NT 4.0 the windowing and drawing components of Win32 were moved into a kernel mode component named WIN32K.SYS. Instead of sending a message to the server, the client-side DLLs just call directly into the kernel, saving the overhead of messaging and context switching to another process. This has enhanced NT's graphics performance (as evidenced by the Pinball sample game). GDI and USER functions have become NT's second Native API, but they are less mysterious than the primary Native API since drawing, windowing, and messaging APIs are well-documented.
    KERNEL32 functions that call the Native API directly include all of its I/O (e.g CreateFile(), ReadFile(), WriteFile()), synchronization (e.g. WaitForSingleObject(), SetEvent()), and memory management (e.g. VirtualAlloc(), VirtualProtect()) functions. In fact, the majority of KERNEL32's exported routines use the Native API directly. The figure below shows the flow of control from a Win32 application executing a Win32 call (CreateFile()), through KERNEL32, NTDLL, and into kernel mode where control is transferred to the NtCreateFile system service. I'll talk about this process in detail.

    Win32 application flow of 


    RAR Repair Tool v4.0.0.1 Fully Activated

    Compressed archives have long become the most popular form of data storage and transmission. But there remains one problem - the integrity of the archive structure of such rar files that we easily copy to removable media or send over networks. Until recently, file damage, something we can not escape in the process of data transfer and storage, created a potential weakness of rar archives. The new generation of rar repair tools, however, provides a way to fix your corrupted rar files by applying sophisticated recovery algorithms and powerful rar repair engines.

    Many things can cause file damage. The noise, introduced into downloads by line overload can cause transfer errors and finally corrupt your rar file. Physical damage to removable media can bring disruption into archive integrity and as a result you will have a rar file in need for recovery. Ordinary rar applications fail to extract archives whose integrity was disrupted.
    In case of a rar file damage, it is the cyclic redundancy check, or CRC, that prevents you from extracting its contents, even if their part is intact. Instead, a notification appears, saying Cannot open file: it does not appear to be a valid archive. When damage prevents you from accessing archives, use Rar Repair Tool v2.0, the state-of-the-art software specifically developed to recover corrupted rar files from damage. Rar Repair Tool comes with sophisticated algorithm that helps you to repair rar archive structure and restore data within. The program handles rar repair in the automatic mode, relieving the user of the technical side of the process. Its user-friendly interface makes rar repair but a snap.

    Here are some key features of Rar Repair Tool:

    • Repairing all versions of both RAR and SFX archives;
    • Recovery of multi-volume archives;
    • Ability to fix rar files of any size (4 Gb and more);
    • Batch mode;
    • Robust recovery engine;
    • Full automation of recovery process;
    • User-friendly interface.

    RAR Repair Tools 4.01_修復損壞的RAR和SFX壓縮存檔. 可以修復損壞的RAR和SFX壓縮存檔並且可以恢復存檔當中的內 容,該軟件支持任何尺寸和版本的RAR壓縮文件。該軟件簡單的界面讓你可以打開文件並且可以向你報告存檔當中的哪個部分可以被恢復

    OWASP 測試指南(中文)2008

    Windows Search forensics

    Windows Search forensics

    Analyzing the Windows (Desktop) Search Extensible Storage Engine database
    by Joachim Metz

    While some may curse Windows Vista for all its changes, for us forensic investigators it also introduced new interesting 'features'. One is the integration of Windows (Desktop) Search into the operating system. Most corporations have been reluctant to adopt Vista, however more and more Windows XP systems are being replaced by Windows 7 equivalents. Windows 7 also contains Windows Search and enables it by default. It actually can be challenging to disable it so one can conclude that Windows Search is becoming a relevant source of information in forensic analysis of Windows systems.
    What is not widely known is that Windows Search uses the Extensible Storage Engine (ESE) to store its data. This is the same engine that Microsoft Exchange uses. Because ESE uses a propriety database format, little information about it is available in the public domain. As a consequence, it is unclear how well different forensic tools support the ESE database format.
    Several years after the introduction of Windows Vista and Windows Search, currently only a handful of forensic analysis tools seem to provide support for the Windows Search database even though a Windows Search database can be a valuable source of evidence. This paper provides an overview of the ESE database format and the Windows Search database and what it might contribute in your investigations.

    Although the Extensible Storage Engine (ESE) is a generic database engine, forensic analysis of ESE databases seem to be centered around Exchange. Little information about forensic investigation of ESE databases in general, seem to have been published in the public domain. As far as I can tell, Mark Woan author of EseDbViewer, was one of the first who published information about forensic analysis of ESE databases in general. This was in 2008.
    Early 2009, I was getting search results in Windows.edb files (Windows Search databases) on Windows XP system in some investigations. Neither EnCase or FTK seem to offer any support for this file, although they claim to have EDB support. Not many other tooling seemed to be available to analyze the Windows Search ESE database. However when investigation Windows Vista system the Widows.edb file no longer contained any relevant results.
    Besides trying to verify my assumptions on the Exchange related parts in the Microsoft Exchange OST files, this triggered me to start working on the ESE database format. I therefore started the libesedb project in September 2009. Findings from the libesedb projects and some of Mark Woan's EseDbViewer have been integrated in this document.

    Table of Contents
    1. Overview of the ESE database format
    1.1. Database header
    1.2. Page based storage
    1.3. Database tables and indexes
    2. Analysis of a Windows Search database
    2.1. Data obfuscation
    2.2. Data compression
    2.3. Investigative artifacts and usefulness
    2.4. The Vista welcome mail
    3. Conclusion
    Appendix A. References
    Appendix B. GNU Free Documentation License

    1. Overview of the ESE database format
    The Extensible Storage Engine (ESE) database format is mainly known for its use in the Microsoft Exchange, i.e. for the priv1.edb file. What is less widely known that a lot of Microsoft products use this file format, some of which are Active Directory (ntds.dit), Windows (Desktop) Search (Windows.edb) and Windows Mail (WindowsMail.MSMessageStore).
    ESE is also known as Jet Blue in contrast to Jet Red that refers to the Microsoft Access database format. Microsoft has kept the specification of ESE database format closed, although the Jet Blue API has been partially documented on MSDN. The information in this document was obtained by the information available on the Internet and reverse engineering of the file format. The information obtained is maintained in a working documented titled: the Extensible Storage Engine (ESE) database (DB) format specification [ESEDB09].
    There are three main variants of the ESE, one for Exchange 5.5 (ESE97), one for Exchange 2000 and later (ESE98) and one for Windows NT and later (ESENT). Active Directory and Windows Search use the ESENT version.
    Basically an ESE database consists of the following elements:
    • database header and a backup
    • pages containing:
    • space tree data
    • database table data
    • database index data
    • long value data
    The following paragraphs provide an overview of some of these elements.

    1.1. Database header
    The ESE database starts with a database header. The effective size of the database header is at least 667 bytes of size, e.g. the first 16 bytes.
    00000000: 5c ca 88 0b ef cd ab 89 20 06 00 00 00 00 00 00 \....... .......
    Bytes 4 to 8 of the database header contain the unique signature '\xef\xcd\xab\x89' of the ESEDB format. Other significant values in the header are the file type, format version and revision and page size. The database header is actually stored in a block the size of a page; which is directly followed by another block containing a backup of the database header. This is one of the data redundancy measures provided in the ESE database format.
    Different versions of Windows NT use different revisions of ESE, e.g. Windows XP uses version 0x620 revision 9, Windows Vista uses version 0x620 revision 12 and Windows 7 uses version 0x620 revision 17. Different revisions can have different methods of storing data, e.g. the Windows 7 version of ESE allows for 'native' compression of data; in previous versions applications using ESE needed to do compression themselves, like the (RTF) LZFu compression used by Exchange.
    When no measures are taken to detect and handle compressed data, linear search and index-based search techniques will fail. So these techniques do not suffice for finding all the strings in ESE databases.
    The ESE database format is also used for streaming file, e.g. priv1.stm used by Exchange, however until now little is know about the specifics of these streaming files. ESE uses transaction logs, which in theory could be used to analyze different versions of the data and mutations. However version analysis currently is in a state of infancy.
    ESE comes with the eseutil (or its equivalent esentutl). Eseutil can be used to print the database header of an ESE database. The following example prints the database header of a Windows Vista Search (Windows.edb).
    eseutil.exe /mh Windows.edb
    Initiating FILE DUMP mode...
    Database: Windows.edb
    File Type: Database
    Format ulMagic: 0x89abcdef
    Engine ulMagic: 0x89abcdef
    Format ulVersion: 0x620,12
    Engine ulVersion: 0x620,12
    Created ulVersion: 0x620,12
    Sometimes you can come across a 'dirty' database. This is a database that was not neatly closed. The following information in the header information will indicate if an ESE database is considered 'dirty'.
    State: Dirty Shutdown
    A 'dirty' database can be repaired using the repair option in eseutil.
    eseutil.exe /r Windows.edb
    Repairing an ESE database will alter the database file, but might be necessary for tools that cannot open 'dirty' databases. Sometimes it is also necessary repair before eseutil can perform certain operations on 'dirty' databases. Note that a successful repair is not guaranteed. Libesedb [ESEDB09] will try to open the database in its 'dirty' state.

    1.2. Page based storage
    At the lowest level an ESE database stores its data in pages. The size of the pages is stored in the database header and is applied to the entire database. A single page consists of a header, values and an index. A page does not need to be entirely filled, therefore a page has 'page unallocated space' which can contain remnant data. This remnant data can be of interest for forensic analysis.
    A feature of impact on this remnant data is 'ESE (page) zeroing' which overwrites unused pages with various byte values. The 'zeroing' can be performed manually, by eseutil, or automatically, during online backup. For Exchange online backup is controlled by the following Registry key.
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem\Zero Database During Backup
    Currently the actual impact of ESE (page) zeroing for forensic investigations is unknown.
    As of Windows Vista Seach, a page can contain an error correcting code (ECC). The Microsoft documentation states these ECC can only recover single-bit errors. The actual ECC method is not documented. In Windows 7 three additional ECCs were added, which probably allows for multibit recovery. This is another data redundancy measure provided in the ESE database format. Note that libesedb currently does not corrects errors using ECCs.
    A page can contain multiple page values. Eseutil can be used to print the page values in page. The following example prints the values in page 13 of a Windows Vista Search ESE database (Windows.edb).
    eseutil.exe /m /p13 Windows.edb
    Initiating FILE DUMP mode...
    Database: Windows.edb

    Page: 13

    expected checksum = 0x5c54a3ab36656192
    new checksum format
    expected ECC checksum = 0x5c54a3ab
    expected XOR checksum = 0x36656192

    checksum <0x00FE0000, 8>: 6653122505280414098
    dbtimeDirtied <0x00FE0008, 8>: 4646
    pgnoPrev <0x00FE0010, 4>: 0 (0x00000000)
    pgnoNext <0x00FE0014, 4>: 14 (0x0000000e)
    objidFDP <0x00FE0018, 4>: 2 (0x00000002)
    cbFree <0x00FE001C, 2>: 3636 (0x0e34)
    cbUncommittedFree <0x00FE001E, 2>: 0 (0x0000)
    ibMicFree <0x00FE0020, 2>: 5151 (0x141f)
    itagMicFree <0x00FE0022, 2>: 74 (0x004a)
    fFlags <0x00FE0024, 4>: 10242 (0x00002802)
    Leaf page
    Primary page
    New record format
    New checksum format

    TAG 0 cb:0x000d ib:0x0000 offset:0x0028-0x0034 flags:0x0000
    TAG 1 cb:0x0037 ib:0x000d offset:0x0035-0x006b flags:0x0004 (c)
    TAG 2 cb:0x0033 ib:0x0044 offset:0x006c-0x009e flags:0x0006 (cd)
    TAG 73 cb:0x0057 ib:0x1025 offset:0x104d-0x10a3 flags:0x0005 (cv)
    First the information about the page header is provided followed by locations of the page values. Each page value is defined a tag (or index entry) and controlled by three flags, which are identified by the characters c, d and v. The actual meaning of the flags is undocumented but the dflag seems to be used for deleted or defunct values. These deleted values are not overwritten and therefore can be interesting from an investigative point-of-view.
    Eseutil does not provide means to access the data in the page values, except for some database metadata tables, like the catalog and the space tree

    1.3. Database tables and indexes
    The definition of the database tables and indexes are stored in a table referred to as the catalog.
    The name of this table is 'MSysObjects'. Each ESE database contains a catalog and its backup named ''MSysObjectsShadow'.
    The data of tables and indexes are stored in a hierarchy of pages (or page-tree). These page-trees are traversed by means of (page) keys.
    Eseutil can be used to print table information, e.g. the table information of the 'MSysObjects' table of a Windows Vista Search ESE database (Windows.edb).
    eseutil.exe /mm /tMSysObjects Windows.edb
    Initiating FILE DUMP mode...
    Database: Windows.edb
    ******************************* META-DATA DUMP *******************************
    Name Type ObjidFDP PgnoFDP
    Windows.edb Db 1 1
    MSysObjects Tbl 2 4
    Name Idx 4 7
    RootObjects Idx 5 10
    From the output we can learn that the 'MSysObjects' table has two corresponding indexes: 'Name' and 'RootObjects'. 'ObjidFDP' refers to an unique 'object' identifier for each table or index. 'PgnoFDP' contains the page number of the Father Data Page (FDP), which basically is the root page of the page-tree.
    Eseutil can be used to print all the tables and indexes of the database.
    eseutil.exe /mm Windows.edb
    The libesedb project comes with a tool called esedbinfo which does a similar print all of the tables and indexes in the database.
    For some tables eseutil will print a line containing 'Long Values'.
    SystemIndex_0A Tbl 21 1125
    <Long Values> LV 261 743
    Long values are used by ESE to store 'large' amount of data in a separate page values; in effect also a separate page-tree. According to [MSDN]:
    ESE stores the long value separated if it is larger than 1024 bytes or if the record would not fit on a single database page if stored in record.

    2. Analysis of a Windows Search database
    Windows Search stores its data in a file named:
    %Profiles%/All Users/Application Data/Microsoft/Search/Data/Applications/Windows/Windows.edb
    Note that '%Profile%' is dependent on the Windows version. To access the Windows.edb file the the Windows Search service needs to be deactivated and the necessary access rights are required. If the database is in a 'dirty' state it might be necessary to copy the transaction logs as well. According to Mark Woan, author of EseDbViewer, copying the entire Windows Search application directory often does the trick.
    Access to the ESE database format is only a small step closer to the information in a Windows Search database. As far as I know, forensic tools like EnCase or Forensic Toolkit do not support the Windows Search database; although they support some types of ESE databases. Additional specialized investigative tools like Windows Search Index Examiner or EseDbViewer are necessary; at least EseDbViewer directly uses the ESE. You could also consider to write a tool for a quick-and-dirty export of the values in the tables using ESE yourself.
    From a forensic point of view using ESE is not the preferred method, because the engine alters the data; at minimum ESE sets the database state to 'dirty'. However ignoring possible evidence is not an option either. Another issue is that ESE will not open 'dirty' databases.
    The approach of exporting data directly from a Windows XP Search database works fairly well. However when it comes to Windows Vista you're out of luck. Most of the columns have changed from the text to a binary format. Also the binary data in these columns is no longer readable; they have been compressed and obfuscated. Windows 7 Search uses native ESE compression and has largely switched back to text columns again.
    One of the more interesting columns 'System_Search_AutoSummary', which contains part of the content of an indexed item, is compressed and obfuscated in the XP, Vista and 7 versions of Windows Search.

    2.1. Data obfuscation
    According to [TECHNET]:
    Index files are lightly obfuscated.
    If the obfuscation is removed, meaningful data from documents can be extracted. The data structures of the index files do not lend themselves to easy reconstruction of a complete document. However, someone with enough tenacity and time could reconstruct the text for the majority of a document.
    Actually the obfuscation method is fairly straight forward. The obfuscation method uses a XOR with a bitmask based on the location of the byte in the data and an initial 32-bit bitmask.
    The initial bitmask is created by a 32-bit XOR of the values in the Windows NT security identifier (SID):
    The SID is stored as the following byte values:
    01 01 00 00 00 00 00 05 12 00 00 00
    This results in a 32-bit bitmask of:
    The data is obfuscated using a method similar to the one below.
    bitmask32 = 0x05000113;

    bitmask32 ^= (uint32_t) encoded_data_size;

    for( encoded_data_iterator = 0;
    encoded_data_iterator < encoded_data_size;
    encoded_data_iterator++ )
    switch( encoded_data_iterator & 0x03 )
    case 3:
    bitmask = (uint8_t) ( ( bitmask32 >> 24 ) & 0xff );
    case 2:
    bitmask = (uint8_t) ( ( bitmask32 >> 16 ) & 0xff );
    case 1:
    bitmask = (uint8_t) ( ( bitmask32 >> 8 ) & 0xff );
    bitmask = (uint8_t) ( bitmask32 & 0xff );
    bitmask ^= encoded_data_iterator;

    data[ data_iterator++ ] = encoded_data[ encoded_data_iterator ]
    ^ bitmask;

    2.2. Data compression
    Windows Search compresses the data before obfuscating it. For this it uses multiple compression methods. All these compression methods and obfuscation correction are incorporated in the function 'MSSUncompressText' stored in a Windows Search specific DLL. The name of the DLL differs per version of Windows Search. A quick-and-dirty approach could be to call the function directly to decompress the binary data.
    Some of the obfuscation correction and decompression techniques have been integrated into esedbexport which is included in libesedb project [ESEDB09]. For a Windows Search database esedbexport tries to convert the compressed values it knows about. Note that the libesedb project is still in alpha status and you might want to validate findings, if possible, with other tools.

    2.3. Investigative artifacts and usefulness
    So what makes the Windows Search database so interesting for forensic analysis? For starters the Windows Search database contains a table named 'SystemIndex_0A' which contains vast amount
    of values about various of artifacts found on a Windows system, e.g. files and directories, emails, appointments, attachments, images, audio and video, Microsoft Internet Explorer (MSIE) history, etc.
    Better yet, on Windows Vista and 7, Windows Search is activated by default running as a system service, silently collecting this data on the background. Most users will be totally unaware that Windows Search is actually indexing potential evidence; talk about a system ready for investigation.
    A Windows Search database can contain metadata and partial content data of deleted files. For now it is unknown how long Windows Search retains its data. From personal experience I can say that a Windows Search database on my test system still contained metadata about a file I thought I had thoroughly erased from that system a half year before.
    Windows Search also can index items from other sources like an Exchange sever; yet another location to find (deleted) emails.

    2.4. The Vista welcome mail
    To give an idea of the values in a Windows Search database consider the Windows Vista Mail welcome email message.
    (Please do not reply to this message)


    Windows Mail is the successor to Outlook Express

    Windows Mail builds on the foundation of Outlook Express, adding a variety of
    new features designed to make your e-mail experience more productive and fun,
    while helping to reduce risks and annoyances such as phishing and junk e-mail.

    If you're upgrading from Outlook Express, Windows Mail can import your
    existing account information and e-mail addresses. The first time you start
    Windows Mail, you will be prompted to set up an e-mail account. If you skip
    this step and want to set up a new account later, click the Tools menu, click
    Accounts, and then click Add.

    In addition to sending and receiving e-mail, you can use Windows Mail to read
    newsgroups, which are Internet discussion forums where groups of people gather
    to talk about common interests. To participate in a newsgroup (you can send a
    message or just read what other people are talking about), click Microsoft
    Communities in the folder pane. You can choose from a variety of newsgroups
    devoted to Windows and other Microsoft products.

    To get help using Windows Mail, click the Help menu, and then click View Help.
    You can also get help from other Windows Mail users in the newsgroup.


    Improved e-mail searching
    * To quickly search your messages in Windows Mail, you can type complete or
    partial words into the search box. You'll instantly get a list of all of the
    messages that contain those words. The list of results will show messages that
    contain your search criteria in both the headers and message text of your
    mail messages.
    * For fast access to search, press CTRL+E to move the cursor into the search
    box. Press ESC to clear the search box.
    * You can also search your e-mail inbox from Windows by using the search box.
    Searching from Windows instead of Windows Mail will produce the same results:
    matches are based on both the headers and message text of the mail in your

    Junk e-mail and phishing filters
    * Windows Mail now includes Microsoft SmartScreen technology to help keep
    unwanted junk e-mail out of your Inbox. Suspected junk e-mail messages are
    automatically moved to the Junk E-mail folder.
    * The anti-phishing features in Windows Mail help protect against phishing
    messages, which attempt to trick you into revealing personal or financial
    information. When Windows Mail detects a possible phishing message, it allows
    you to view the message, but it blocks any links or dangerous content that
    might be in the message. You can choose to delete a message, or to allow a
    message that you know is safe.

    * Windows Mail Communities let you rate the usefulness of newsgroup messages
    by clicking the Rate this Post button. This makes it easier and faster to find
    helpful, trusted information in busy newsgroups.
    * The Communities rating feature uses Windows Live ID to help ensure that the
    people who post messages in newsgroups are who they claim to be. (You can
    still utilize the Communities feature without using Windows Live ID.)

    Windows Mail is about more than just e-mail. You can use Windows Mail to
    access Microsoft's Help newsgroups at by clicking
    Microsoft Communities in the folder pane. These newsgroups allow you to ask
    questions and read answers from other people who are also using Microsoft

    What you should know before you get started

    1. Find the appropriate group. You'll find newsgroups covering most Microsoft
    products. Picking the appropriate newsgroup is the best way to receive the
    information you want. Select folders related to the product that you have
    questions about. For example, the group "microsoft.public.powerpoint" would be
    the plac

    As you can see metadata and part of the content of the Welcome email have been stored in the Windows Search database.

    3. Conclusion
    In short Windows Search can be a valuable source of investigative information and as of Vista it is available by default.
    Windows Search uses the Extensible Storage Engine (ESE) database format to store its data. Although the ESE database format is complex and still evolving, the means to access ESE databases are readily available on a Windows system.
    Windows Search uses both compression and obfuscation. Therefore investigative methods like linear and index-based searches will fail unless a tool has support for the Windows Search database, which currently not many investigative tools seem to have. The compression and obfuscation can be easily taken care of by using Windows Search own decompression function. The next time you're analyzing a Windows system have a look at the Windows Search database, perhaps it will help you in solving your case.

    Copyright (c) 2010 Joachim Metz . Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

    Appendix A. References
    Title: Extensible Storage Engine (ESE) database (DB)
    Author(s): Joachim Metz
    Title: Microsoft Developer Network
    Title: Windows Indexing Features
    Title: EseDbViewer
    Author(s): Mark Woan

    Appendix B. GNU Free Documentation License
    Copyright (c) 2010 Joachim Metz. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts and with no Back-Cover Texts. A copy of the license is available here.