Issue with "Unix" on APFS file systems
Moderators: Cat_7, Ronald P. Regensburg, ClockWise
- Ronald P. Regensburg
- Expert User
- Posts: 7832
- Joined: Thu Feb 09, 2006 10:24 pm
- Location: Amsterdam, Netherlands
Issue with "Unix" on APFS file systems
There appears to be an issue with "Unix" on APFS file systems (Mojave, Catalina, and some High Sierra systems). The issue affects both SheepShaver and BasiliskII.
When files are copied in SheepShaver to "Unix", they loose their resource fork.
I suppose it was not noticed earlier because most users will move files that rely on the resource fork from the host into SheepShaver rather than from SheepShaver to the host. File transfers from the host into SheepShaver via shared folder/Unix work OK.
For a workaround one can use Disk Utility to create an empty HFS+ disk image. The image can be a sparsebundle image. If mounted before the emulator is launched, the image volume can be used as shared "folder". In preferences, the "Unix root" path would be: /Volumes/name-of-image-volume
When files are copied in SheepShaver to "Unix", they loose their resource fork.
I suppose it was not noticed earlier because most users will move files that rely on the resource fork from the host into SheepShaver rather than from SheepShaver to the host. File transfers from the host into SheepShaver via shared folder/Unix work OK.
For a workaround one can use Disk Utility to create an empty HFS+ disk image. The image can be a sparsebundle image. If mounted before the emulator is launched, the image volume can be used as shared "folder". In preferences, the "Unix root" path would be: /Volumes/name-of-image-volume
- adespoton
- Forum All-Star
- Posts: 4277
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: Issue with "Unix" on APFS file systems
Any chance someone can update the Unix code to either use AppleDouble or named forks when writing out? An option to choose between the two would be awesome
That way, we aren't depending on deprecated file write methods. This may also fix the "can't run/open files directly from Unix drive" issue as a side benefit
That way, we aren't depending on deprecated file write methods. This may also fix the "can't run/open files directly from Unix drive" issue as a side benefit
Re: Issue with "Unix" on APFS file systems
extfs_macosx.cpp and extfs_unix.cpp use different methods of storing resource forks.
extfs_macosx.cpp uses the named fork to store the resource fork data: /path/file/..namedfork/rsrc
See https://github.com/cebix/macemu/blob/ma ... x.cpp#L226
extfs_unix.cpp uses helper files to store the resource fork data: /path/.rsrc/file
See https://github.com/kanjitalk755/macemu/ ... ix.cpp#L75
It would be nice to merge the two files together and also add fallback handling.
It may be possible that the reason for not being able to open files directly from Unix drive is that some File System Manager functions are unimplemented. See https://github.com/cebix/macemu/blob/ma ... fs.cpp#L21
extfs_macosx.cpp uses the named fork to store the resource fork data: /path/file/..namedfork/rsrc
See https://github.com/cebix/macemu/blob/ma ... x.cpp#L226
extfs_unix.cpp uses helper files to store the resource fork data: /path/.rsrc/file
See https://github.com/kanjitalk755/macemu/ ... ix.cpp#L75
It would be nice to merge the two files together and also add fallback handling.
It may be possible that the reason for not being able to open files directly from Unix drive is that some File System Manager functions are unimplemented. See https://github.com/cebix/macemu/blob/ma ... fs.cpp#L21
- adespoton
- Forum All-Star
- Posts: 4277
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: Issue with "Unix" on APFS file systems
Makes sense. I know the issue isn't with named forks themselves, as I've been using named forks with APFS for a few years now.
[edit] If someone wants to create a fallback system, I'd be more than happy to create a tool that could rebuild HFS-style files or dsk images from their three parts. It'd be strongly related to this: https://github.com/adespoton/kaitai-applebackup
[edit] If someone wants to create a fallback system, I'd be more than happy to create a tool that could rebuild HFS-style files or dsk images from their three parts. It'd be strongly related to this: https://github.com/adespoton/kaitai-applebackup
- Ronald P. Regensburg
- Expert User
- Posts: 7832
- Joined: Thu Feb 09, 2006 10:24 pm
- Location: Amsterdam, Netherlands
Re: Issue with "Unix" on APFS file systems
For some time we use here only BasiliskII and SheepShaver SDL2 ports from https://github.com/kanjitalk755/macemu
Asked about this issue, kanjitalk755 replied:
Asked about this issue, kanjitalk755 replied:
Someone here who could and would contribute to kanjitalk755 source to solve this issue?I am not familiar with the file systems, so it is difficult to solve the issue.
-
- Apple Corer
- Posts: 239
- Joined: Thu Nov 09, 2017 12:06 pm
Re: Issue with "Unix" on APFS file systems
I found the difference between HFS+ and APFS that caused this issue.
https://news.ycombinator.com/item?id=20629003
https://github.com/kanjitalk755/macemu
I recommend to back up the shared folder for testing.
https://news.ycombinator.com/item?id=20629003
And fixed.On HFS/HFS+, file/..namedfork/rsrc will always exist (and usually be empty). on APFS, file/..namedfork/rsrc will generate an ENOENT error if the resource fork/attribute doesn't exist.
https://github.com/kanjitalk755/macemu
I recommend to back up the shared folder for testing.
- Ronald P. Regensburg
- Expert User
- Posts: 7832
- Joined: Thu Feb 09, 2006 10:24 pm
- Location: Amsterdam, Netherlands
Re: Issue with "Unix" on APFS file systems
Quick test with SheepShaver build in Mojave with one file copied both ways seems to work. No time today for more testing. I will post the test build later.
Should it work the same way with BasiliskII?
Will it still also work on a HFS+ host?
Should it work the same way with BasiliskII?
Will it still also work on a HFS+ host?
Last edited by Ronald P. Regensburg on Sun Jan 12, 2020 2:33 pm, edited 1 time in total.
Reason:
Reason:
-
- Apple Corer
- Posts: 239
- Joined: Thu Nov 09, 2017 12:06 pm
Re: Issue with "Unix" on APFS file systems
Yes.Ronald P. Regensburg wrote:Should it work the same way with BasiliskII?
Yes.Ronald P. Regensburg wrote:Will it still also work on a HFS+ host?
- Ronald P. Regensburg
- Expert User
- Posts: 7832
- Joined: Thu Feb 09, 2006 10:24 pm
- Location: Amsterdam, Netherlands
Re: Issue with "Unix" on APFS file systems
I tested Unix/shared folder with new builds of both BasiliskII (System 7.5.5, HFS) and SheepShaver (MacOS 9.0.5, HFS+) on Mojave 10.14.6 with APFS.
I never tested this as extensively on earlier hosts with HFS+, so I cannot tell if the behaviour is different.
Copying from the host to the guest leaves files intact in every respect.
Copying from the guest to 'Unix' changes creation and modification dates to the date and time the files were copied.
After copying back from 'Unix' to the guest, apart from the creation and modification dates, the files appear unchanged, resource fork, creator and type, Finder flags, etc. Also, the files behave as expected, whether document files or application files.
Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
After dismissing the message by clicking OK, the resources open in ResEdit as expected. After these "minor repairs", again opening in ResEdit does not again show that message. I have no clue about what was 'repaired'.
I see no difference in behaviour of the files between before and after the "repairs" by ResEdit.
I did not see any adverse effects on the shared folder or its content.
(As always, I advise not to use folders with important content, like the Documents folder, as shared folder.)
I posted the two test applications:
SheepShaver: https://www.emaculation.com/sheepshaver ... s_test.zip
BasiliskII: https://www.emaculation.com/basilisk/Ba ... s_test.zip
Note that the applications are not notarized, so you will need to use right-click (or control-click) to open them the first time.
Unless someone finds serious bugs, I will post notarized builds later.
I never tested this as extensively on earlier hosts with HFS+, so I cannot tell if the behaviour is different.
Copying from the host to the guest leaves files intact in every respect.
Copying from the guest to 'Unix' changes creation and modification dates to the date and time the files were copied.
After copying back from 'Unix' to the guest, apart from the creation and modification dates, the files appear unchanged, resource fork, creator and type, Finder flags, etc. Also, the files behave as expected, whether document files or application files.
Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
After dismissing the message by clicking OK, the resources open in ResEdit as expected. After these "minor repairs", again opening in ResEdit does not again show that message. I have no clue about what was 'repaired'.
I see no difference in behaviour of the files between before and after the "repairs" by ResEdit.
I did not see any adverse effects on the shared folder or its content.
(As always, I advise not to use folders with important content, like the Documents folder, as shared folder.)
I posted the two test applications:
SheepShaver: https://www.emaculation.com/sheepshaver ... s_test.zip
BasiliskII: https://www.emaculation.com/basilisk/Ba ... s_test.zip
Note that the applications are not notarized, so you will need to use right-click (or control-click) to open them the first time.
Unless someone finds serious bugs, I will post notarized builds later.
-
- Forum All-Star
- Posts: 1726
- Joined: Tue Oct 14, 2008 12:12 am
Re: Issue with "Unix" on APFS file systems
And I've updated all my projects with the new code:
SheepShaver Wrapper: http://www.columbia.edu/~em36/sheepshaverwrapper.html
And these, which you'll need to search for, because direct links to them are forbidden in this forum:
MacOS9 for macOS
WPMacApp
System761
System755
I've also updated the "rootless" BasiliskII version that I posted on another thread.
SheepShaver Wrapper: http://www.columbia.edu/~em36/sheepshaverwrapper.html
And these, which you'll need to search for, because direct links to them are forbidden in this forum:
MacOS9 for macOS
WPMacApp
System761
System755
I've also updated the "rootless" BasiliskII version that I posted on another thread.
-
- Apple Corer
- Posts: 239
- Joined: Thu Nov 09, 2017 12:06 pm
Re: Issue with "Unix" on APFS file systems
I looked into resource forks copied to HFS+ and APFS.Ronald P. Regensburg wrote:Still, apparently something changed: When I open files that had been copied to 'Unix' in ResEdit, ResEdit shows this message: "This file needed minor repairs and is now OK."
As a result, length of a resource fork on APFS seems to be 512 bytes boundary.
Extras were padded with zeros.
ResEdit does not support this specification, so it may have been determined to be broken.
-
- Apple Corer
- Posts: 239
- Joined: Thu Nov 09, 2017 12:06 pm
Re: Issue with "Unix" on APFS file systems
A resource fork is written with an integer multiple of 512 (extfs.cpp#L1933).
Then it is truncated to the correct length (extfs.cpp#L1750).
It is truncated correctly on HFS+, but left it at the originally written length on APFS.
The solution is currently unknown.
Then it is truncated to the correct length (extfs.cpp#L1750).
It is truncated correctly on HFS+, but left it at the originally written length on APFS.
The solution is currently unknown.
- Ronald P. Regensburg
- Expert User
- Posts: 7832
- Joined: Thu Feb 09, 2006 10:24 pm
- Location: Amsterdam, Netherlands
Re: Issue with "Unix" on APFS file systems
Thanks for the explanation. So the "minor repair" that ResEdit makes is probably truncating the additional zeros.
I can see no difference in behaviour between files before and after they were copied to 'Unix', so I suppose this difference in resource fork size is harmless.
I posted new notarized builds (dated 12 January 2020) for both SheepShaver and BasiliskII.
I can see no difference in behaviour between files before and after they were copied to 'Unix', so I suppose this difference in resource fork size is harmless.
I posted new notarized builds (dated 12 January 2020) for both SheepShaver and BasiliskII.