Recently, we improved OFRAK Python package and dependency handling, resulting in quicker installation of more functionality. These improvements are available now on PyPI across ofrak, ofrak_type, ofrak_io, and ofrak_patch_maker.
Wrapping an existing tool in an OFRAK Component is a relatively common pattern. After all, OFRAK does not intend to “reinvent the wheel” if other teams have already shared a good tool for the job at hand. However, this can mean that a component isn’t ready to use after a simple “pip install.” Often an extra step or two might be needed, something like (for a Debian Linux user) “apt install -y special_program” for the SpecialProgramAnalyzer Component to run successfully.
NOTE: For researchers on a longer-lived investigative effort or for teams using OFRAK in a Continuous Integration (CI) pipeline that needs reproducible results, the provided Docker image recipe builds the Red Balloon Security-recommended environment. This includes all components’ dependencies. If you find yourself installing OFRAK and its dependencies multiple times, consider using the Docker image for a consistent, pre-built environment.
When we built OFRAK, we considered it reasonable to separate those components with heavy dependencies and limited applications from the more lightweight set of core software, for instance, packages that need to handle various archives as opposed to simple Python-based byte string modification of a file. Now that we’ve added some tools to make these dependencies easier to understand, install, or ignore, we are merging ofrak_core and ofrak_components.
Now, when developers and researchers pip install ofrak they will get access to more components than they did before, right off the bat. Many of them are wrappers for external tools, and most of those wrappers are only for handling specific file types. For instance, the CpioUnpacker wraps usage of the cpio utility. If the user isn’t working with a CPIO filesystem, that unpacker won’t run, and it won’t matter if the utility binary is not installed on the system $PATH.
If the user does happen to run into a CPIO file during an invocation of unpack_recursively, OFRAK will fail trying to unpack it. Two choices are provided:
ofrak.component.abstract.ComponentMissingDependencyError: Missing cpio tool needed for CpioUnpacker!
E apt installation: apt install cpio
E brew installation: brew install cpio
E See https://www.gnu.org/software/cpio/ for more info and installation help.
E Alternatively, OFRAK can ignore this component (and any others with missing dependencies) so that they will never be run: OFRAK(..., exclude_components_missing_dependencies=True)
The user can either:
1. Install it
Or
2. Specify that OFRAK ignores this component and any others missing dependencies. This is the quickest way to unblock OFRAK and keep going if the dependency really isn’t needed.
>>> o = OFRAK(exclude_components_missing_dependencies=True)
>>> async def main(ofrak_context):
...
>>> o.run(main)
[ofrak_context.py: 185] Skipped registering the following components due to missing dependencies: CpioPacker, CpioUnpacker. Run `python3 -m ofrak deps --missing-only` for more details.
Without CpioUnpacker, OFRAK won’t be able to unpack the CPIO file. If you don’t need to unpack that file, then excluding the CpioUnpacker in this case is fine.
If a user would like to ensure all necessary binary dependencies are installed (and still does not want to use Docker) they can use a new command-line interface (CLI) tool to list and check for them.
% python -m ofrak list
ofrak
ApkIdentifier
ApkPacker
ApkUnpacker
MemoryRegionProgramAttributesAnalyzer
BinaryExtendModifier
BinaryPatchModifier
BinwalkAnalyzer
Bzip2Packer
Bzip2Unpacker
Md5Analyzer
Sha256Analyzer
AddCommentModifier
DeleteCommentModifier
CpioFilesystemAnalyzer
CpioPacker
CpioUnpacker
DeviceTreeBlobIdentifier
DeviceTreeBlobPacker
DeviceTreeBlobUnpacker
DtbHeaderAnalyzer
% python -m ofrak deps
[ ] binwalk
https://github.com/ReFirmLabs/binwalk
[BinwalkAnalyzer]
[ ] 7z
https://p7zip.sourceforge.net/
[P7zUnpacker, P7zPacker]
[ ] zip
https://linux.die.net/man/1/zip
[ZipPacker]
[ ] pigz
https://zlib.net/pigz/
[GzipUnpacker, GzipPacker]
[✓] tar
https://www.gnu.org/software/tar/
[TarUnpacker, TarPacker]
[ ] zstd
http://facebook.github.io/zstd/
[ZstdPacker, ZstdUnpacker]
[✓] unzip
https://linux.die.net/man/1/unzip
[ApkIdentifier, ZipUnpacker]
[ ] apktool
https://ibotpeaches.github.io/Apktool/
[ApkUnpacker, ApkPacker]
[ ] lzop
https://www.lzop.org/
[LzoUnpacker, LzoPacker]
[ ] cpio
https://www.gnu.org/software/cpio/
[CpioPacker, CpioUnpacker]
[ ] /usr/local/bin/uber-apk-signer.jar
https://github.com/patrickfav/uber-apk-signer
[ApkPacker]
[ ] entropy.so.1
None
[DataSummaryAnalyzer]
[ ] unar
https://theunarchiver.com/command-line
[RarUnpacker]
[ ] java
https://openjdk.org/projects/jdk/11/
[ApkPacker]
[ ] mksquashfs
https://github.com/plougher/squashfs-tools.git
[SquashfsPacker]
[ ] unsquashfs
https://github.com/plougher/squashfs-tools.git
[SquashfsUnpacker]
On a fresh system many of these will be missing. If the user can leverage a package manager like apt (on Ubuntu) or brew (on OS X) they can install most of them with one line:
$ python3 -m ofrak deps –-missing-only –-packages-for brew | xargs brew install -y
(replacing brew with apt when on an Ubuntu machine)
Checking again however, they will find some are still missing….
% python3 –m ofrak deps –-missing-only
[ ] entropy.so.1
None
[DataSummaryAnalyzer]
[ ] unsquashfs
https://github.com/plougher/squashfs-tools.git
[SquashfsUnpacker]
[ ] /usr/local/bin/uber-apk-signer.jar
https://github.com/patrickfav/uber-apk-signer
[ApkPacker]
[ ] mksquashfs
https://github.com/plougher/squashfs-tools.git
[SquashfsPacker]
[ ] binwalk
https://github.com/ReFirmLabs/binwalk
[BinwalkAnalyzer]
[ ] apktool
https://ibotpeaches.github.io/Apktool/
[ApkPacker, ApkUnpacker]
These tools have a more involved installation process, and users should consult each one for the exact instructions. Of course, at this point users can tell which tools each Component depends on, so they can verify the script will run without choking on forgotten dependencies.
To review, our recommended steps are:
Internally, the main impact we’ve observed from these updates is that our security researchers are able to get up and running even faster for an investigation or one-off binary mod. We hope you see these gains too!
Cheers to the New Year and all your binary resolutions from our team in NYC!
Sal Stolfo was an original founding member of Red Balloon Security, Inc.
© 2022 Red Balloon Security.
All Rights Reserved.
Contact us now to discover more about Red Balloon Security’s range of solutions and services or to arrange a demonstration.
Reach out to learn more about our embedded security offering and to schedule a demo.
Reach out to learn more about our embedded security offering and to schedule a demo.
Reach out to learn more about our embedded security offering and to schedule a demo.
Reach out to learn more about our embedded security offering and to schedule a demo.
