Issue with "Unix" on APFS file systems

About SheepShaver, a PPC Mac emulator for Windows, MacOS X, and Linux that can run System 7.5.3 to MacOS 9.0.4.

Moderators: Cat_7, Ronald P. Regensburg, ClockWise

Post Reply
User avatar
Ronald P. Regensburg
Expert User
Posts: 7821
Joined: Thu Feb 09, 2006 10:24 pm
Location: Amsterdam, Netherlands

Issue with "Unix" on APFS file systems

Post by Ronald P. Regensburg »

There appears to be an issue with "Unix" on APFS file systems (Mojave, Catalina, and some High Sierra systems). The issue affects both SheepShaver and BasiliskII.

When files are copied in SheepShaver to "Unix", they loose their resource fork.

I suppose it was not noticed earlier because most users will move files that rely on the resource fork from the host into SheepShaver rather than from SheepShaver to the host. File transfers from the host into SheepShaver via shared folder/Unix work OK.

For a workaround one can use Disk Utility to create an empty HFS+ disk image. The image can be a sparsebundle image. If mounted before the emulator is launched, the image volume can be used as shared "folder". In preferences, the "Unix root" path would be: /Volumes/name-of-image-volume
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: Issue with "Unix" on APFS file systems

Post by adespoton »

Any chance someone can update the Unix code to either use AppleDouble or named forks when writing out? An option to choose between the two would be awesome :)

That way, we aren't depending on deprecated file write methods. This may also fix the "can't run/open files directly from Unix drive" issue as a side benefit :)
uyjulian
Apple Corer
Posts: 244
Joined: Fri Aug 27, 2010 1:02 am

Re: Issue with "Unix" on APFS file systems

Post by uyjulian »

extfs_macosx.cpp and extfs_unix.cpp use different methods of storing resource forks.

extfs_macosx.cpp uses the named fork to store the resource fork data: /path/file/..namedfork/rsrc
See https://github.com/cebix/macemu/blob/ma ... x.cpp#L226

extfs_unix.cpp uses helper files to store the resource fork data: /path/.rsrc/file
See https://github.com/kanjitalk755/macemu/ ... ix.cpp#L75

It would be nice to merge the two files together and also add fallback handling.

It may be possible that the reason for not being able to open files directly from Unix drive is that some File System Manager functions are unimplemented. See https://github.com/cebix/macemu/blob/ma ... fs.cpp#L21
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: Issue with "Unix" on APFS file systems

Post by adespoton »

Makes sense. I know the issue isn't with named forks themselves, as I've been using named forks with APFS for a few years now.

[edit] If someone wants to create a fallback system, I'd be more than happy to create a tool that could rebuild HFS-style files or dsk images from their three parts. It'd be strongly related to this: https://github.com/adespoton/kaitai-applebackup
User avatar
Ronald P. Regensburg
Expert User
Posts: 7821
Joined: Thu Feb 09, 2006 10:24 pm
Location: Amsterdam, Netherlands

Re: Issue with "Unix" on APFS file systems

Post by Ronald P. Regensburg »

For some time we use here only BasiliskII and SheepShaver SDL2 ports from https://github.com/kanjitalk755/macemu

Asked about this issue, kanjitalk755 replied:
I am not familiar with the file systems, so it is difficult to solve the issue.
Someone here who could and would contribute to kanjitalk755 source to solve this issue?
kanjitalk755
Apple Corer
Posts: 239
Joined: Thu Nov 09, 2017 12:06 pm

Re: Issue with "Unix" on APFS file systems

Post by kanjitalk755 »

I found the difference between HFS+ and APFS that caused this issue.

https://news.ycombinator.com/item?id=20629003
On HFS/HFS+, file/..namedfork/rsrc will always exist (and usually be empty). on APFS, file/..namedfork/rsrc will generate an ENOENT error if the resource fork/attribute doesn't exist.
And fixed.

https://github.com/kanjitalk755/macemu

I recommend to back up the shared folder for testing.
User avatar
Ronald P. Regensburg
Expert User
Posts: 7821
Joined: Thu Feb 09, 2006 10:24 pm
Location: Amsterdam, Netherlands

Re: Issue with "Unix" on APFS file systems

Post by Ronald P. Regensburg »

Quick test with SheepShaver build in Mojave with one file copied both ways seems to work. No time today for more testing. I will post the test build later.

Should it work the same way with BasiliskII?

Will it still also work on a HFS+ host?
Last edited by Ronald P. Regensburg on Sun Jan 12, 2020 2:33 pm, edited 1 time in total.
Reason:  
kanjitalk755
Apple Corer
Posts: 239
Joined: Thu Nov 09, 2017 12:06 pm

Re: Issue with "Unix" on APFS file systems

Post by kanjitalk755 »

Ronald P. Regensburg wrote:Should it work the same way with BasiliskII?
Yes.
Ronald P. Regensburg wrote:Will it still also work on a HFS+ host?
Yes.
User avatar
Ronald P. Regensburg
Expert User
Posts: 7821
Joined: Thu Feb 09, 2006 10:24 pm
Location: Amsterdam, Netherlands

Re: Issue with "Unix" on APFS file systems

Post by Ronald P. Regensburg »

I tested Unix/shared folder with new builds of both BasiliskII (System 7.5.5, HFS) and SheepShaver (MacOS 9.0.5, HFS+) on Mojave 10.14.6 with APFS.

I never tested this as extensively on earlier hosts with HFS+, so I cannot tell if the behaviour is different.

Copying from the host to the guest leaves files intact in every respect.

Copying from the guest to 'Unix' changes creation and modification dates to the date and time the files were copied.

After copying back from 'Unix' to the guest, apart from the creation and modification dates, the files appear unchanged, resource fork, creator and type, Finder flags, etc. Also, the files behave as expected, whether document files or application files.

Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
After dismissing the message by clicking OK, the resources open in ResEdit as expected. After these "minor repairs", again opening in ResEdit does not again show that message. I have no clue about what was 'repaired'.

I see no difference in behaviour of the files between before and after the "repairs" by ResEdit.

I did not see any adverse effects on the shared folder or its content.
(As always, I advise not to use folders with important content, like the Documents folder, as shared folder.)

I posted the two test applications:
SheepShaver: https://www.emaculation.com/sheepshaver ... s_test.zip
BasiliskII: https://www.emaculation.com/basilisk/Ba ... s_test.zip

Note that the applications are not notarized, so you will need to use right-click (or control-click) to open them the first time.

Unless someone finds serious bugs, I will post notarized builds later.
emendelson
Forum All-Star
Posts: 1706
Joined: Tue Oct 14, 2008 12:12 am

Re: Issue with "Unix" on APFS file systems

Post by emendelson »

And I've updated all my projects with the new code:

SheepShaver Wrapper: http://www.columbia.edu/~em36/sheepshaverwrapper.html

And these, which you'll need to search for, because direct links to them are forbidden in this forum:

MacOS9 for macOS
WPMacApp
System761
System755

I've also updated the "rootless" BasiliskII version that I posted on another thread.
kanjitalk755
Apple Corer
Posts: 239
Joined: Thu Nov 09, 2017 12:06 pm

Re: Issue with "Unix" on APFS file systems

Post by kanjitalk755 »

Ronald P. Regensburg wrote:Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
I looked into resource forks copied to HFS+ and APFS.
As a result, length of a resource fork on APFS seems to be 512 bytes boundary.
Extras were padded with zeros.
ResEdit does not support this specification, so it may have been determined to be broken.
kanjitalk755
Apple Corer
Posts: 239
Joined: Thu Nov 09, 2017 12:06 pm

Re: Issue with "Unix" on APFS file systems

Post by kanjitalk755 »

A resource fork is written with an integer multiple of 512 (extfs.cpp#L1933).
Then it is truncated to the correct length (extfs.cpp#L1750).
It is truncated correctly on HFS+, but left it at the originally written length on APFS.

The solution is currently unknown.
User avatar
Ronald P. Regensburg
Expert User
Posts: 7821
Joined: Thu Feb 09, 2006 10:24 pm
Location: Amsterdam, Netherlands

Re: Issue with "Unix" on APFS file systems

Post by Ronald P. Regensburg »

Thanks for the explanation. So the "minor repair" that ResEdit makes is probably truncating the additional zeros.

I can see no difference in behaviour between files before and after they were copied to 'Unix', so I suppose this difference in resource fork size is harmless.

I posted new notarized builds (dated 12 January 2020) for both SheepShaver and BasiliskII.
Post Reply