TAPI 1500.0.12.3 source has been published #32
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
miles/apple-libtapi#32
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
See https://github.com/apple-oss-distributions/tapi/tree/tapi-1500.0.12.3
1400.0.11 is also available https://github.com/apple-oss-distributions/tapi/tree/tapi-1400.0.11
TAPI 1600.0.11.8 sources have been released.
so i was feeling adventurous and tried:
git remote add -f libtapi-oss path/to/apple-libtapi-oss.git
git remote update
git diff 1300.6.5 tapi-1600.0.11.8
and i almost fainted by how long the differences are. its 5 am now.
i'm guessing i can't just "copy and paste" the new tapi branch in the src/tapi location.
Merging TAPI isn't easy... It also requires updating LLVM and applying patches/changes again. That's why I haven't done it so far.
Is there anything important missing in 1300.6.5?
A better strategy would be to diff this repo's tapi with Apple's tapi 1300 (https://github.com/apple-oss-distributions/tapi), and diff this repo's LLVM with https://github.com/swiftlang/llvm-project/tree/apple/stable/20210107, then apply those changes to tapi 1600 and https://github.com/swiftlang/llvm-project/releases/tag/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-07-02-a
Maybe change the repo to include apple llvm/api as submodules and apply some patches before actually building. Might make updating easier.
I don't think there's anything too massive, I can still use the latest macOS SDK with ld64 linked with tapi 1300. I'm the kind of person who can't stand having an old version number though, I might try to update it myself later, I think I've tried before and failed but another try couldn't hurt.
Progress update: the build is including incompatible .inc files from my host system's LLVM installation. Not sure how to make it build its own .inc files. I think if I solved that issue it would be basically done.
Porting/building TAPI is a mess... You are only building TAPI not the whole LLVM tree right?
I was able to successfully build TAPI 1600.0.11.8 by first building the
clangBasicandvt_gentargets, thenlibtapi. What a mess. Also had to remove a lot of cmake stuff in order to make it build.I added the 1600.0.11.8 sources.
x86_64h is kind of broken now, like a fallback to x86_64 is missing. Not sure if it is the TAPI library or the linker that's at fault.
Everything seems to work. Would be really interesting to know if x86_64h is also broken in Xcode.
Too bad I don't own a single Apple macOS device...
I replaced the llvm, clang, and tapi sources with tapi 1600 and the llvm commit that the tapi readme said to use. Also had to add the LLVM cmake stuff which gets it's own folder now. Only change I made to the build.sh script was removing the include fix, since the directories it was including don't exist in the newer LLVM sources. Building vt_gen along with clangBasic did the trick for me too.
I own 5 but one runs Linux, two run 10.5, one runs 10.6, the last is a haswell iMac I could probobly make run the latest macOS with OCLP, I'll do that and test. Maybe you should setup CI with a macOS github runner, github gives unlimited minutes to public repos now. You could easily compare the behavior of the ports with the Xcode versions.
Oop, ld64 fails to link with it. I'll try your version.
Your version worked, can confirm x86_64h isn't working, I'm installing macOS 14 on my iMac now.
x86_64h is not broken with Xcode 16.2.
You forgot to remove
-DCMAKE_CXX_FLAGS="$INCLUDE_FIX"by the wayReplacing the Xcode libtapi.dylib with one built from the unmodified open source release of tapi 1600 results in the same issue with x86_64h
I don't know enough about the tapi internals to fix the issue, Since it's fixed in Xcode it'll probably be fixed with tapi 1700. In the mean time I'll try merging tapi 1500 to at least get a more recent version that still works.
I merged tapi 1500 and it works better, but still not perfect, spamming the terminal about x86_64h not being present in tbds
It does link an executable though. I'll try 1400 next.
I can't do 1400 because I can't find the correct LLVM.
1500 is available at https://github.com/Un1q32/apple-libtapi/tree/1500.0.12.3 if that's useful to anyone.
That's good enough. It seems they don't care about x86_64h anymore. Maybe we shouldn't either...
I mean the only thing x86_64h does differently from x86_64 is enabling optimizations for core-avx2.
Are you using ld64? Could you try 16.3?
Happens with both ld64 and the new ld. 16.3 updated the LLVM stuff to 1700 so it would be incompatible with tapi 1600. I could try 16.0 (the first version with LLVM 1600) later.
I'll see if I can maybe silence the attempting fallback warning. Hopefully they'll open source 1700 soon.
Thank you. You don't have to do any further testing, unless you want to play around a bit more. :)
I extracted the latest Xcode 16.4 Beta.
ld64 is 955.13 ✅
cctools is 1024.3 ✅
TAPI is 1700.0.3.5 (as you already mentioned)
Dang it. Why didn't they open source TAPI 1700 already. :(
Please open a pull request for 1500.
For the warning part: I'd lke to have an option to silence the warning with an env variable and keep it enabled by default.
FYI you can go here to see the versions, though it doesn't show the ld64 version for Xcode versions after 15.0 (when the new linker was introduced) https://en.wikipedia.org/wiki/Xcode#Toolchain_versions
They usually wait until the next release to open source the previous, though it's not exact, pretty much whenever they feel like it. Sometimes they'll wait even longer if the source has secret stuff, like I think the ld64 907 stuff took a lot longer because it had VisionOS code.
The warning is from ld64, not TAPI. My current plan is to check the version of TAPI from ld64 and only disable the warning with TAPI 1500. The warn shouldn't happen if there isn't a fallback available, so it shouldn't have any bad side effects.
Hmm, ld64 with TAPI 1500 is taking a LOT longer to link a binary than with 1300, ~80 ms to link hello world for x86_64 on my machine to ~200 ms with 1500, 300 ms when linking x86_64h (x86_64h is still 80 with 1300).
Thanks. Good to know.
Yeah, it's sometimes completely random. It's like they're rolling dice to decide when to open-source things.
Fine 👍
Weird. :/ Does this only affect 1500 or also 1600?
It does not affect 1600 (back to ~80 ms), so that's good news for the eventual 1700 open sourcing.
Also I believe x86_64h was never really meant to be it's own architecture, but instead just a slice in a universal binary that supported systems could use instead of the standard slice. This is supported by the fact that the Intel bits of modern macOS versions are compiled with AVX2 for the last few versions, but don't use x86_64h. This is probably why Apple doesn't care about it much, since it's meant to be a niche thing for people who want to support both haswell and pre-haswell CPUs but still want the AVX2 performance benefits, but all pre-haswell CPUs are over a decade old, so most developers who want AVX2 will just enable AVX2 in the default x86_64 slice and not support pre-haswell.
Great!
Yes, makes absolutely sense.
Here's a patch for cctools-port that silences the warn when using TAPI 1500. I'm not bothering with an actual PR because the performance regressions in 1500 make it basically unusable for me anyway. For now I'll see if I can maybe fix the bug causing x86_64h issues in 1600 myself, since I don't want to wait potentially months for Apple to open source TAPI 1700.
If Xcode 16.3 ships with TAPI 1700 and Apple published source for the 16.3 toolset is TAPI 1600, that should be reported in Feedback Assistant. Otherwise you'll have to wait however long until they decide to publish source for Xcode 16.4 (which is still beta at the moment, but should be released today?)
That would be great if you could figure out what's causing it. I tried to
grepforx86_64stuff in the TAPI sources yesterday but couldn't find anything suspicous.They probably won't push the source today, the open source releases lag roughly 1 toolchain update behind Xcode, and Apple only updates their toolchain twice a year (the x.0 and x.3 Xcode releases). The open source releases lagging behind Xcode seems to be intentional, not an accident, or at least they have to do some kind of review of the source to make sure it doesn't have any secrets before they can release it, and that review takes roughly a few months.
and yet I keep telling them this is a violation of the GPLv2 because Xcode ships with a modified version of git that they aren't providing the sources for... (and they keep ignoring me and releasing code whenever they feel like) 🙃
Wouldn't they only have to provide the source for their modified git on time? I don't believe they link any code from git in cctools.
Oh wait oh shit new ld64 and cctools just dropped what the fuck.
tapi will probobly drop soon then
Modern cctools doesn't use any GPL code AFAIK, though I know it used tolooks like there's still a little in otool's classic disassembler. I remember Saurik was complaining to Apple about their GPL violations as far back as 2008 (https://www.saurik.com/toolchain.html).Though unless you can fund your own lawsuit or get someone with a lot of influence like Linus Torvalds to complain, you're probably out of luck.
I found the LLVM for TAPI 1400 (https://github.com/swiftlang/llvm-project/tree/swift-DEVELOPMENT-SNAPSHOT-2021-10-26-a). Attempting a 1400 port to see if we can at least get a little upgrade without any issues before the 1700 source is released.
I'm giving up again, getting stuck on
ninja: error: unknown target 'libtapi', did you mean 'lib/all'?. Looks like tapi isn't getting recognized as a subproject, maybe I have the wrong LLVM still?My work is at https://github.com/Un1q32/apple-libtapi/tree/1400.0.11 if anyone wants to try and finish it.