TL;DR:
1.29 -> 1.30 seemed to stabilise frametimes, 1.29 -> 1.31 screwed up slowest frames and introduced many spikes which can results in stuttering. Your mileage may vary. More people should benchmark and test. Visual results in an imgur link mid-way post.
Remember the flood of optimisation patch requests a while back? Well, with version 1.30 (Ashe release patch) we got a lot of under-the-hood internal reworks as shown by the necessary full reinstallation of the game. But was it an/the optimization patch people were waiting for?
There are many indicators which should be studied for this: game launching times, map loading times, occurrences of character models and textures not loading properly, input lag, network lag, framerates etc.
I personally hadn’t experienced the big problems (“framerates halved” or something) many people were complaining about on the subreddit, but I did some tests anyway.
Relevant testing hardware
- ASUS ROG PG278QR monitor overclocked to 165 Hz; both G-Sync and Fast sync enabled
- 8GB KFA2 GeForce GTX 1070 EX OC Sniper Edition running at PCIe 3.0 16x
- Intel Core i7-8700K running at 3,70 GHz (non-overclocked)
- Z370 Gigabyte Aorus Gaming 7-OP
- Kingston 2x8 GB DDR4-2666 (non-overclocked)
- Windows Server 2016 Datacenter
- OS booted and Overwatch running from 2 x Optane 32 GB NVMes in M.2 RAID 0. (Apparently not fully utilising all processors PCIe lanes since 16 already go to GPU according to GPU-Z utility.)
Now it is quite obvious that with these components the game is not running super slow in the first place and some potential bottlenecks due to programming might be overcome by brute force, but timeframe analysis can show stability of those frames. Also note that my testing was unsuitable for investigating, among other things, map loading times.
Methodology. I chose several different moments from the game in different versions (1.29, 1.30 and 1.31) for which I recorded frame times for 60 seconds each. I used FRAPS 3.5.99 for the benchmarking and FRAFS Bench Viewer 0.3.0.3 for the analysis. No video was recorded; only the times needed for the system to render each frame. Following scenarios were benchmarked and analysed:
- 1.29 First 60 seconds (from when spawn doors open) of defence on Route 66 with framecap of 163 (from game settings)
- 1.31 First 60 seconds (from when spawn doors open) of defence on Route 66 (163)
- 1.29 First 60 seconds (from when spawn doors open) of Ilios Well (left spawn from seaview) with framecap of 300
- 1.30 First 60 seconds (from when spawn doors open) of Ilios Well (left spawn from seaview) (300)
- 1.29 Menu screen of Horizon in the background (163)
- 1.29 Menu screen of Horizon in the background (300)
- 1.30 Menu screen of Horizon in the background (163)
- 1.30 Menu screen of Horizon in the background (300)
To reduce error source, all games were normal quick plays (no custom games). I would’ve had more pairs to test if only I had actually got the same maps but for example it happened that during the Ashe patch I never, by chance, got to play Route 66. Not once before 1.31 was already live. During menu tests, no settings/career menus were visited. In gameplay tests, I always played the same hero in the beginning. The framerate cap was the only game setting to change for the tests, otherwise here are the visual settings:
System memory was the biggest hardware bottleneck and care was taken to not run anything extra while Overwatch and test software were running. (Browsers closed etc.)
For those not familiar with benchmarking with frametimes instead of frames per second (FPS), it’s a bit more deep/true metric, because FPS is by its nature a ratio of two quantities and you can get all kinds of FPS values based on the time window you’re calculating it from. Since FPS is always an average over some time, it can mask a lot of variability in the rendering. When you obtain frametimes, you get all the data available and you can see if there are any hiccups (causing (micro)stuttering) that wouldn’t show up in FPS numbers; a high average FPS alone is not okay if the frametimes keep going up and down a lot. When looking at frametime graphs, more steady is better for the graph should be as uniform and horizontal as possible. A common framerate of 60 translates to 1/60 ≈ 16,7 milliseconds, and with this setup I cropped and scaled the graph to only show frametimes less than that. If you do a similar benchmark but are expecting lower average framerates, then do of course prepare for higher frametimes. Especially with lower framerates, frametime stability is something you’ll want over few frames per second more on average.
Results.
(Links below are brutalised because of that horribly counter-productive “Sorry, you can’t include links in your posts” restriction! Don’t blame me.)
Pictorial results in an Imgur album:
imgur .com/gallery/WGdnJrH
Full data files (Dropbox share link) if someone wants to view them in more detail or do more analysis:
dropbox .com/s/g99m1wypdq518vl/OW_benchmarks.zip?dl=0
I did not rigorously record, but I did not notice nor numerically see any network lag or input lag difference (closest which could be SIM number in OW debug network information overlay).
Conclusions.
Not going for null hypotheses or confidence intervals this time, sorry.
As expected menu performance was not affected due to the already heavy hard framecap of 60 FPS anytime you’re in the main menu. It’s a console-like solution to make sure everything is steady; limiting framerates ensures everything runs smoother without hiccups.
Between 1.29 and 1.30 — the patch that actually required a full reinstallation (the game didn’t ask for your permission) — it seems Ilios didn’t really change a lot. There are no noticeable differences in lag spikes, average frametimes of FPS although it does seem like the longest-taking frames (0,1 % got a little buff) hitched a tiny bit higher. … nothing I could sincerely attribute to the patch though; could just be that some visually heavy hero wasn’t played during the second benchmark. (This is where it would start being useful to do these benchmarks multiple times, but I couldn’t wait to get the same maps again and again.) Although the frametime graph does show somewhat more consistent in 1.30 i.e. times seem to be more together instead of spread out around the average.
So: at least on some high-end desktop and on Ilios, the 1.30 patch didn’t cause noticeable/measurable rendering lag. Consistency might even got better, but wouldn’t call it a big optimisation. If the effect was much larger on lower-end desktops, then it’s amazing and hopefully intended. 
Comparison of 1.29 and 1.31, though, is more alarming. In the 1.31 graph, there are many more lag spikes visible. Average framerates or frametimes didn’t really change, but the slowest-drawn 0,1 % of frames became even slower to render! The percentile-defining frames took now 1,31/8,45≈ 1,55 … 55 percent more time to draw! That’s bad. Slowest framerates took a dive as well. I don’t know what could’ve caused that but I’m not blaming a specific random hero or visual effects for that anymore. :<