In the JS world, tracing was abandoned because it didn't scale to real world code.
JS JITs (the production ones, like JSC's) have no such thing as trace blockers that prevent the surrounding code from being optimized. You might have an operation (like a call to some wacky native function) that is itself not optimized, but that won't have any impact on the JIT's ability to optimize the code surrounding that operation.
Tracing is just too much of a benchmark hack overall IMO. Tracing would only be a good idea in a world where it's too expensive to run a real optimizing JIT. But the JS, Java, and .NET experiences show that a real optimizing JIT - with all of its compile time costs - is exactly what you want because it results in predictable speed-ups
marky19911 hour ago
Pypy's tracing JIT has worked pretty well for years, so that doesn't seem universal. I admit I have fairly low expectations on a JIT succeeding (in any meaningful fashion) in cpython given the constraints that it currently has, but I'm generally a skeptic, so maybe i'm being overly pessimistic.
pizlonator1 hour ago
What does “worked pretty well” really mean though?
When we talk about JS or Java JITs working well, we are making statements based on intense industry competition where if a JIT had literally any shortcoming then a competitor would highlight it in competitive benchmarking and blog posts. So, the competition forced aggressive improvements and created a situation where the top JITs deliver reliable perf across lots of workloads.
OTOH PyPy is awesome but just hasn’t had to face that kind of competitive challenge. So we probably can’t know how far off from JS JITs it is.
One thing I can say is when I compared it to JSC by writing the same benchmark in both Python and JS, JSC beat it by 4x or so.
acdha55 minutes ago
I think the Java JITs are a better comparison because the workload is more similar: JavaScript is weird for how it’s expected to start in a fraction of a second and soak up a huge bolus of code which may substantially never be used whereas most of the performance-sensitive Python code stabilizes quickly and loads what it uses really early on.
cogman1011 minutes ago
The Java JIT and most other Javascript jits are essentially operating the same way. The core difference is the java language spec sets up a whole bunch of requirements that need to be figured out at startup and are easy to trigger.
For example, static initialization on classes. The JDK has a billion different classes and on startup a not insignificant fraction of those end up getting loaded for all but the simplest applications.
Essentially, Java and the JS jits are both initially running everything interpreted and when a hot method is detected they progressively start spending the time sending those methods and their statistics to more aggressive JIT compilers.
A non-insignificant amount of time is being spent to try and make java start faster and a key portion of that is resolving the class loading problem.
IainIreland19 minutes ago
I don't know how JSC handles it, but in SM `eval` has significant negative effects on surrounding code. (We also decline to optimize functions containing `with` statements, but that's less because it's impossible and more because nobody uses them.)
cogman109 minutes ago
Last I saw (and I admit this is pretty dated) V8 was doing the same thing. try/catch at one point in V8 would cause the surrounding method to be deoptimized.
In the JS world, tracing was abandoned because it didn't scale to real world code.
JS JITs (the production ones, like JSC's) have no such thing as trace blockers that prevent the surrounding code from being optimized. You might have an operation (like a call to some wacky native function) that is itself not optimized, but that won't have any impact on the JIT's ability to optimize the code surrounding that operation.
Tracing is just too much of a benchmark hack overall IMO. Tracing would only be a good idea in a world where it's too expensive to run a real optimizing JIT. But the JS, Java, and .NET experiences show that a real optimizing JIT - with all of its compile time costs - is exactly what you want because it results in predictable speed-ups
Pypy's tracing JIT has worked pretty well for years, so that doesn't seem universal. I admit I have fairly low expectations on a JIT succeeding (in any meaningful fashion) in cpython given the constraints that it currently has, but I'm generally a skeptic, so maybe i'm being overly pessimistic.
What does “worked pretty well” really mean though?
When we talk about JS or Java JITs working well, we are making statements based on intense industry competition where if a JIT had literally any shortcoming then a competitor would highlight it in competitive benchmarking and blog posts. So, the competition forced aggressive improvements and created a situation where the top JITs deliver reliable perf across lots of workloads.
OTOH PyPy is awesome but just hasn’t had to face that kind of competitive challenge. So we probably can’t know how far off from JS JITs it is.
One thing I can say is when I compared it to JSC by writing the same benchmark in both Python and JS, JSC beat it by 4x or so.
I think the Java JITs are a better comparison because the workload is more similar: JavaScript is weird for how it’s expected to start in a fraction of a second and soak up a huge bolus of code which may substantially never be used whereas most of the performance-sensitive Python code stabilizes quickly and loads what it uses really early on.
The Java JIT and most other Javascript jits are essentially operating the same way. The core difference is the java language spec sets up a whole bunch of requirements that need to be figured out at startup and are easy to trigger.
For example, static initialization on classes. The JDK has a billion different classes and on startup a not insignificant fraction of those end up getting loaded for all but the simplest applications.
Essentially, Java and the JS jits are both initially running everything interpreted and when a hot method is detected they progressively start spending the time sending those methods and their statistics to more aggressive JIT compilers.
A non-insignificant amount of time is being spent to try and make java start faster and a key portion of that is resolving the class loading problem.
I don't know how JSC handles it, but in SM `eval` has significant negative effects on surrounding code. (We also decline to optimize functions containing `with` statements, but that's less because it's impossible and more because nobody uses them.)
Last I saw (and I admit this is pretty dated) V8 was doing the same thing. try/catch at one point in V8 would cause the surrounding method to be deoptimized.