FreeBSD's 1.6 JDK on Mac OS X

04 Nov 2007, 21:52 PST

NOTE: You probably want to check out the latest updates

Introduction

I've long wondered what it would take to get the FreeBSD Java Port running on OS X, so this weekend I spent a couple days getting Java 1.6 running on my x86 Leopard machine.

Weekend is over, and I can report partial success -- hotspot compiles, the jre mostly bootstraps, and Hello World runs. Anything complex appears to trigger stack alignment issues (Apple's i386 API requires a 16-byte aligned stack)

landonf@max:javasrc_1_6_jrl/control/build/bsd-i586> ./bin/java -version
java version "1.6.0_03-p3"
Java(TM) SE Runtime Environment (build 1.6.0_03-p3-landonf_04_nov_2007_00_06-b00)
Java HotSpot(TM) Server VM (build 1.6.0_03-p3-landonf_04_nov_2007_01_30-b00, mixed mode)
landonf@max:javasrc_1_6_jrl/control/build/bsd-i586> ./bin/java Hello
Hello World!

What's missing?

Missing pieces:

Additionally, there's no Aqua support -- Swing would require X11.

Source? Compiling?

The patch is based on the BSD 1.6 JDK patchset, which means you need to download Sun's JDK source, the FreeBSD patchset, AND the Darwin patches. Instructions on downloading the Sun source, and the FreeBSD patchset, are available from the FreeBSD JDK site

The Darwin patchset should be applied on top of the BSD patchset. By downloading this source code, you certify that you are a Licensee in good standing under the Java Research License ("License") of the Java(tm) 2 SDK, and that your access, use and distribution of code and information you may obtain at this site is subject to the License. Download patch-jdk16-apple-alpha1.

Compilation requires OpenMotif, Nawk, and possiblity additional software -- I installed all the dependencies using MacPorts. To build, cd to control/make and run the following (tweaked appropriately):

env DYLD_BIND_AT_LAUNCH=1 \
DYLD_LIBRARY_PATH=<javasrcpath>/control/build/bsd-i586/lib/i386/client \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
DONT_BUILD_DEPLOY="YES" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=4 PARALLEL_BUILD_JOBS=4 

The build does not complete (yet), but a bootstrap JVM is built (and can be run from) ./control/build/bsd-i586/bin/java

Patch for CVE-2007-2788: Java Image Parsing Code Buffer Overflow

14 Aug 2007, 00:20 PDT

Four months ago, Chris Evans, of Google's Security Team, released an advisory regarding a heap overflow in Sun's Java ICC (image) profile parsing code. (CVE-2007-2788). The release was cordinated with Sun, and an updated Java release (JDK 1.5.0_11-b03) was made available for Window, Solaris, and Linux.

Apple's Java runtime has not yet been updated, so I've gone ahead and written a run-time patch for my own use. If you'd like to use the patch too, you can download the source, or a pre-built binary. You'll need to install Application Enhancer to use the patch. Alternatively, you could simply disable Java in your browser to close the most likely vector.

The issue is due to an integer overflow that occurs when validating that an ICC header tag does not exceed the total length of the heap allocated profile data buffer; The comparison will overflow if the header declares an too-large tag size (See section 7.1 of the ICC.1:2004-10 specification for more information on the header tag format).

For a proof of concept, I've uploaded my regression pages here -- fair warning -- this link will crash an unpatched browser.

Update

Google Group's file hosting was giving Safari users trouble, so I'm now hosting the run-time patch locally.

Installing the iPhone Toolchain using MacPorts

12 Aug 2007, 15:32 PDT

NOTE: This post has been archived for historical purposes. The toolchain has advanced considerably, and Apple is planning to release their own SDK. I plan to hold out on further development, toolchain or otherwise, until it's released.

Please see iPhone Toolchain Project for up-to-date instructions.

To facilitate my own iPhone development, I've committed three new ports for the iphone-binutils project to MacPorts. Once installed, you're ready to compile Hello, World. Prior to installation, you'll need to acquire a copy of the iPhone root disk image ("Heavenly") and install its contents in /opt/local/arm-apple-darwin/heavenly. The image is required to provide the necessary libraries for linking cross-compiled iPhone binaries, and can't be re-distributed.

Extracting and Installing the iPhone Libraries

To start, download and decompress iPhone1,1_1.0_1A543a_Restore.ipsw:

 user@host> curl -O http://appldnld.apple.com.edgesuite.net/content.info.apple.com/iPhone/061-3538.20070629.B7vXa/iPhone1,1_1.0_1A543a_Restore.ipsw
 user@host> unzip iPhone1,1_1.0_1A543a_Restore.ipsw \*.dmg

This will extract two disk images: "694-5259-38.dmg" and "694-5262-39.dmg". The encrypted "694-5262-39.dmg" disk image contains the iPhone root. The decryption key for this image is stored in plain text within the "asr" binary on the corresponding "694-5259-38.dmg". To retrieve the key, run the following:

 user@host> strings 694-5259-38.dmg| grep "^[0-9a-fA-F]*$" | awk '{ if (length($1) == 72) print; }'

This should output a 72 character hex string, which you'll use as the decryption key.

In order to perform the decryption, you'll need modified 'vfdecrypt' -- a command utility for decrypting Mac OS X disk images. The source is available here. To compile, simply type "make" in the vfdecrypt-iphone directory. The provided version of vfdecrypt was slightly modified to support direct input of the private AES and SHA1 HMAC keys -- these are normally wrapped with a user-supplied passphrase (via 3DES-EDE), which is not available. vfdecrypt was written by Ralf-Philipp Weinmann, Jacob Appelbaum, and Christian Fromme.

Once you've build vfdecrypt, use it to decrypt the disk image:

 user@host> ~/vfdecrypt-iphone/vfdecrypt -i 694-5262-39.dmg -k <hex key> -o heavenly.dmg 

Now, mount the disk image and copy the contents to /opt/local/arm-apple-darwin/heavenly:

 user@host> open heavenly.dmg
 user@host> sudo mkdir -p /opt/local/arm-apple-darwin/heavenly
 user@host> (cd /Volumes/Heavenly1A543a.UserBundle && tar cf - .) | (cd /opt/local/arm-apple-darwin/heavenly && sudo tar xvf -)

Installing the Toolchain

To install the toolchain:

 sudo port install arm-apple-darwin-runtime

You should now be able to compile standard Unix software:

 CC=arm-apple-darwin-cc CPP=llvm-cpp ./configure --host=arm-apple-darwin

For more information on compiling your first GUI application, check out the UIKit Hello World

MoAB Fixes Net a Bug

19 Apr 2007, 13:35 PDT

Today Apple released Security Update 2007-004. The update includes quite a few important fixes, and also includes a fix for a bug I stumbled across during the Month of Apple Bugs, while regression testing the patch for the Quicktime RTSP URL Handling Buffer Overflow Vulnerability.

While testing the RTSP fix, I had experimented with providing long HTTP URLs to the QuickTime Plugin, and caused a crash. At the time, I mistakenly assumed that the two bugs were the same -- it wasn't until after the RTSP issue was fixed that I looked more closely and submitted the issue to Apple.

In the end, Apple did all the hard work in tracking down the bug to Libinfo. They were reasonably communicative on status, and provided the opportunity to regression test their fix prior to release.

Amazon S3 Backups (and a java S3 library)

10 Apr 2007, 19:21 PDT

Introduction

At Three Rings, we use Bacula as a disk-based backup system. We originally implemented off-site backups by rsync'ing snapshots to an out-of-state server, but as the size of our backups headed towards terabytes, we started looking for a solution that didn't require installing a remote multi-terabyte RAID.

Amazon's Simple Storage Service (S3) provides access to Amazon's "highly scalable, reliable, fast, inexpensive data storage infrastructure" -- In brief, S3 provides a limitless amount of internet-accessible storage, accessed by a simple REST (or SOAP) API.

As a part of our (Three Rings) next project, Whirled, I implemented a relatively complete Java library for S3's REST API. Having the code on hand, it made sense to leverage both the code, and S3, for our off-site backups.

Streaming Data to (and from) S3

To back up multiple terabytes of data remotely, I had three requirements:

The result is S3Pipe, a small utility which implements piping of streams to and from S3:

landonf:s3lib> echo "hello, world" | s3pipe ... upload --stream helloworld
landonf:s3lib> s3pipe ... download --stream helloworld
hello, world

In other words, S3Pipe supports reading data from stdin and writing it to S3, as well as reading data from S3 and writing it to stdout. The implementation:

I hope that others will assist in incrementally improving the feature set; built-in public-key encryption & signing, simple HMAC support, even more robust transient/network failure recovery.

Download & Use

Both the S3 client library and S3Pipe are available under the BSD license, as a 1.0-beta1 release. This is new software, and I strongly recommend testing any backups -- that said, the code has nearly 100% unit test coverage, has been tested locally, and is in active production use. The primary reason for the beta designator is that I still haven't written documentation!

You can download the library source here (pgp signature). To build s3lib and s3pipe, you will need the Java JDK (1.5+) and Ant (1.7+, available via MacPorts). Inside the source directory, run:

ant dist

This will create dist/s3lib.jar and dist/s3pipe.jar. S3Pipe is a stand-alone jar -- it can be copied anywhere once built. To execute S3Pipe (and print command usage), run:

java -jar dist/s3pipe.jar

S3Pipe requires you to provide your AWS id and key in a seperate properties file. See 'test.properties.example' as an example.

If you'd like javadocs for s3lib, run:

ant javadoc

However, be aware that the API is still subject to change.

S3Pipe in action: S3 Dump and Restore

All of our servers at Three Rings run FreeBSD, and so I decided to use dump(8), with its' built-in support for UFS2 file system snapshots, to dump file system snapshots from our primary Bacula server to Amazon S3.

The dump command makes use of "-P", or "pipe command". When provided a -P option, dump(8) will execute the given sh script, and pipe its output to the child's stdin. The following command line is a big one: it dumps a snapshot of the /export file system, piping the output through gzip for compression, gpg for symmetric encryption, throttle to rate limit the output to 20 Mbps, and S3Pipe, to write the stream to S3.

/sbin/dump -a -P "gzip -c | gpg --symmetric --batch --passphrase-file key.txt | /usr/local/bin/throttle -m 20 \
| java -jar s3pipe.jar --keyfile s3secrets.properties --bucket example.backup.bucket --stream bacula-/export upload" -0Lu /export

Restoration is, of course, almost the same thing in reverse.

java -jar s3pipe.jar --keyfile s3secrets.properties --bucket example.backup.bucket --stream bacula-/export download |\
gpg --decrypt --batch --passphrase-file key.txt | gzip -cd | restore

Hiring a Lead Web Engineer

02 Apr 2007, 10:05 PDT

I run the Engineering Infrastructure Team at Three Rings. We're an engineering-focused IT group, and we're responsible for everything from FreeBSD kernel development to development of our billing transaction/gateway system. We maintain several open source projects of our own, as well as donating time to external projects -- I implemented Bacula's network encryption (TLS) support for our use here at Three Rings.

We're a small company, and I have a strong personal investment in our success; I'd like to hire a lead web engineer who will bring solid engineering practices to our web sites and applications.

For more information, and if you'd like to apply, the job ad is posted here. Also, please feel free to send me any questions.

MoAB Fixes Update for Security Update 2007-003 and Mac OS X 10.4.9

01 Apr 2007, 13:49 PDT

Howdy All. I've updated the plugin for 10.4.9/Security Update 2007-003. This update removes a number of patches for issues fixed by the Software Update:

The following patches remain:

You can download the binary release, or the source. GPG signatures are available for the binary and the source.

Month of Apple Bugs - Patch 27

28 Jan 2007, 21:42 PST

Today's patch includes fixes for three different issues:

Thanks to my friends William Carrel and Rosyna of Unsanity for their work on tracking down the bugs and assistance in implementing the patches.

You can download the source, or a pre-built binary. As always, you'll need Application Enhancer to use the patches.

Update 9:00 1/29/06: Modified the patch (to 27-1) to match the latest 10.4.8 version of the PowerPC Software Update.app, 2.0.5 (Intel is 2.0.4).

This release also removes a few patches:

Flip4Mac WMV Vulnerability

The ever-handy Flip4Mac WMV QuickTime components are susceptable to a buffer overflow due to an integer overflow when comparing two buffer lengths as signed values. The comparison results in a stack-based buffer overflow that is potentially remotely exploitable by a malicious WMV movie. Both Mail.app and Safari will inline WMV movies (I accidentally crashed Mail.app when I tried to attach a proof-of-concept to an e-mail I'd drafted).

This patch is more complex; nearly all symbols are stripped in Flip4Mac, so finding the vulnerable code requires locating a public symbol and then computing the offset to the vulnerable code, for both PowerPC and Intel. Due to the relatively short length of the function in question, I decided to completely re-implement it, using proper unsigned comparisons. You can peruse the implementation in Google Code's web interface.

Software Update Catalog Filename Format String Vulnerability

Software Update falls prey to a common format string vulnerability, passing the user-supplied file name to an alert panel. However, I'm not aware of any means of delivering a maliciously named file without requiring direct user interaction. The error dialog in question is only displayed when a file without the "sucatalog" extension is passed to Software Update (eg, "swutmp") -- the patch checks file extension, and escaps any '%' characters if the extension is not "sucatalog".

Installer Package Filename Format String Vulnerability

Installer is also susceptable to the same issue -- when an error is encountered while opening a file, Installer will pass the user-supplied file name to an alert panel. A zipped installer package will be automatically opened by Safari if "Open Safe Files" is enabled. The patch works by "context patching" NSRunAlertPanel() while within -[InstallerController openFile:withOptions:]. The guarded NSRunAlertPanel() passes it's msg argument as a vararg argument to the "%@" format string, and the patch is removed when -[InstallerController openFile:withOptions:] exits. A reference count is used to handle recursion.

Month of Apple Bugs - Almost February ...

26 Jan 2007, 12:59 PST

Soon the month will draw to a close, and with it, The Month of Apple Bugs. Lest you think I've dropped off the map, I've set aside the weekend for addressing the new issues. Apologies for the delay -- it's been a busy work week!

I'm also pleased to announce a number of software updates for outstanding issues. The turn-around time on these updates is admirable.

Panic's Transmit - MoAB #19

The indubitably responsive developers at Panic have released an update to fix the ftp:// and ftps:// buffer overflow announced in the MoAB #19 advisory.

Rumpus FTP Server - MoAB #18

The good-natured fellows of Maxum Development Corporation have released an update to Rumpus, fixing the issues outlined in MoAB #18. Existing Rumpus 5.1 users may use the "Software Updates" feature in Rumpus to automatically download and install the update. All others can download the updated demo application.

0-Day Patches

Where a critical high-risk "0-day" issue exists, and is practical to patch, I plan on continuing to provide patches beyond the Month of Apple Bugs. The Java GIF Patch is the first of these "non-MoAB" fixes.

The MOAB Fixes group remains the primary point of coordination, and I expect to move this effort to a more official home than my own web log.

Month of Apple Bugs - Day 20

21 Jan 2007, 12:58 PST

The latest MoAB issue is a format string vulnability in iChat. The proof of concept uses Safari's auto-handling of aim:// URLs to pass a 'aim:gochat?roomname' request to iChat.

The patch for the issue was adeptly implemented by William Carrel. His patch works by guarding -[ActiveChat showError: (NSString *) error] and escaping any format string specifiers in the error message. As always, patches are coordinated at the MOAB Fixes group if you'd like to participate.

You can download the binary, or the source. As always, you'll need Application Enhancer to use the patches.

Java Patch Update - Firefox

This release also updates the patch for the Java GIF Heap Overflow issue. The update expands the library path regexes to ensure that Firefox-launched JVMs are always patched, in addition to Safari.

Rumpus FTP Server - MoAB #18

The expeditious fellows over at Maxum Development also have a fix in the works for the MoAB #18 issue in Rumpus, and will be releasing an update in short order. Cheers for their fast turn-around on the issue.