Page 1 of 1

Issue with "Unix" on APFS file systems

Posted: Wed Dec 04, 2019 4:34 pm
by Ronald P. Regensburg
There appears to be an issue with "Unix" on APFS file systems (Mojave, Catalina, and some High Sierra systems). The issue affects both SheepShaver and BasiliskII.

When files are copied in SheepShaver to "Unix", they loose their resource fork.

I suppose it was not noticed earlier because most users will move files that rely on the resource fork from the host into SheepShaver rather than from SheepShaver to the host. File transfers from the host into SheepShaver via shared folder/Unix work OK.

For a workaround one can use Disk Utility to create an empty HFS+ disk image. The image can be a sparsebundle image. If mounted before the emulator is launched, the image volume can be used as shared "folder". In preferences, the "Unix root" path would be: /Volumes/name-of-image-volume

Re: Issue with "Unix" on APFS file systems

Posted: Wed Dec 04, 2019 4:54 pm
by adespoton
Any chance someone can update the Unix code to either use AppleDouble or named forks when writing out? An option to choose between the two would be awesome :)

That way, we aren't depending on deprecated file write methods. This may also fix the "can't run/open files directly from Unix drive" issue as a side benefit :)

Re: Issue with "Unix" on APFS file systems

Posted: Fri Dec 06, 2019 6:28 pm
by uyjulian
extfs_macosx.cpp and extfs_unix.cpp use different methods of storing resource forks.

extfs_macosx.cpp uses the named fork to store the resource fork data: /path/file/..namedfork/rsrc
See https://github.com/cebix/macemu/blob/ma ... x.cpp#L226

extfs_unix.cpp uses helper files to store the resource fork data: /path/.rsrc/file
See https://github.com/kanjitalk755/macemu/ ... ix.cpp#L75

It would be nice to merge the two files together and also add fallback handling.

It may be possible that the reason for not being able to open files directly from Unix drive is that some File System Manager functions are unimplemented. See https://github.com/cebix/macemu/blob/ma ... fs.cpp#L21

Re: Issue with "Unix" on APFS file systems

Posted: Fri Dec 06, 2019 7:56 pm
by adespoton
Makes sense. I know the issue isn't with named forks themselves, as I've been using named forks with APFS for a few years now.

[edit] If someone wants to create a fallback system, I'd be more than happy to create a tool that could rebuild HFS-style files or dsk images from their three parts. It'd be strongly related to this: https://github.com/adespoton/kaitai-applebackup

Re: Issue with "Unix" on APFS file systems

Posted: Sat Dec 07, 2019 11:05 am
by Ronald P. Regensburg
For some time we use here only BasiliskII and SheepShaver SDL2 ports from https://github.com/kanjitalk755/macemu

Asked about this issue, kanjitalk755 replied:
I am not familiar with the file systems, so it is difficult to solve the issue.
Someone here who could and would contribute to kanjitalk755 source to solve this issue?

Re: Issue with "Unix" on APFS file systems

Posted: Sun Jan 12, 2020 11:43 am
by kanjitalk755
I found the difference between HFS+ and APFS that caused this issue.

https://news.ycombinator.com/item?id=20629003
On HFS/HFS+, file/..namedfork/rsrc will always exist (and usually be empty). on APFS, file/..namedfork/rsrc will generate an ENOENT error if the resource fork/attribute doesn't exist.
And fixed.

https://github.com/kanjitalk755/macemu

I recommend to back up the shared folder for testing.

Re: Issue with "Unix" on APFS file systems

Posted: Sun Jan 12, 2020 2:32 pm
by Ronald P. Regensburg
Quick test with SheepShaver build in Mojave with one file copied both ways seems to work. No time today for more testing. I will post the test build later.

Should it work the same way with BasiliskII?

Will it still also work on a HFS+ host?

Re: Issue with "Unix" on APFS file systems

Posted: Mon Jan 13, 2020 1:34 am
by kanjitalk755
Ronald P. Regensburg wrote:Should it work the same way with BasiliskII?
Yes.
Ronald P. Regensburg wrote:Will it still also work on a HFS+ host?
Yes.

Re: Issue with "Unix" on APFS file systems

Posted: Mon Jan 13, 2020 2:27 pm
by Ronald P. Regensburg
I tested Unix/shared folder with new builds of both BasiliskII (System 7.5.5, HFS) and SheepShaver (MacOS 9.0.5, HFS+) on Mojave 10.14.6 with APFS.

I never tested this as extensively on earlier hosts with HFS+, so I cannot tell if the behaviour is different.

Copying from the host to the guest leaves files intact in every respect.

Copying from the guest to 'Unix' changes creation and modification dates to the date and time the files were copied.

After copying back from 'Unix' to the guest, apart from the creation and modification dates, the files appear unchanged, resource fork, creator and type, Finder flags, etc. Also, the files behave as expected, whether document files or application files.

Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
After dismissing the message by clicking OK, the resources open in ResEdit as expected. After these "minor repairs", again opening in ResEdit does not again show that message. I have no clue about what was 'repaired'.

I see no difference in behaviour of the files between before and after the "repairs" by ResEdit.

I did not see any adverse effects on the shared folder or its content.
(As always, I advise not to use folders with important content, like the Documents folder, as shared folder.)

I posted the two test applications:
SheepShaver: https://www.emaculation.com/sheepshaver ... s_test.zip
BasiliskII: https://www.emaculation.com/basilisk/Ba ... s_test.zip

Note that the applications are not notarized, so you will need to use right-click (or control-click) to open them the first time.

Unless someone finds serious bugs, I will post notarized builds later.

Re: Issue with "Unix" on APFS file systems

Posted: Mon Jan 13, 2020 2:54 pm
by emendelson
And I've updated all my projects with the new code:

SheepShaver Wrapper: http://www.columbia.edu/~em36/sheepshaverwrapper.html

And these, which you'll need to search for, because direct links to them are forbidden in this forum:

MacOS9 for macOS
WPMacApp
System761
System755

I've also updated the "rootless" BasiliskII version that I posted on another thread.

Re: Issue with "Unix" on APFS file systems

Posted: Tue Jan 14, 2020 1:50 am
by kanjitalk755
Ronald P. Regensburg wrote:Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
I looked into resource forks copied to HFS+ and APFS.
As a result, length of a resource fork on APFS seems to be 512 bytes boundary.
Extras were padded with zeros.
ResEdit does not support this specification, so it may have been determined to be broken.

Re: Issue with "Unix" on APFS file systems

Posted: Tue Jan 14, 2020 7:09 am
by kanjitalk755
A resource fork is written with an integer multiple of 512 (extfs.cpp#L1933).
Then it is truncated to the correct length (extfs.cpp#L1750).
It is truncated correctly on HFS+, but left it at the originally written length on APFS.

The solution is currently unknown.

Re: Issue with "Unix" on APFS file systems

Posted: Tue Jan 14, 2020 3:43 pm
by Ronald P. Regensburg
Thanks for the explanation. So the "minor repair" that ResEdit makes is probably truncating the additional zeros.

I can see no difference in behaviour between files before and after they were copied to 'Unix', so I suppose this difference in resource fork size is harmless.

I posted new notarized builds (dated 12 January 2020) for both SheepShaver and BasiliskII.