Pico4 performance testing

November 4, 2022

I would like to see relative performance on Pico4 with latest supported drivers and across GPU and CPU architectures - sort of a general test for VR users to see are there major discrepancies.

I tested my configuration x570, 5800x, 32gb DDR4 3800Mhz, 6900XT water cooled, latest windows10 22H2 19045.2193, latest AMD Chipset drivers, latest and VD recommended AMD Radeon drivers.

my Pico4 is connected like this wirelessly and the AP router is setup exactly like this post from @SCG_Fenris_Wolf

I test like this, put my headset on, start virtual desktop inside of it with a controller, then doubleclick the simple batch file il2.bat (whole content is: "C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\IL-2 Sturmovik Great Battles\bin\game\Il-2.exe" ), load SYN_VANDER benchmark mission, start it, remove the headset and put it on table, then reset the view to default.

I took @chiliwili69 test mission as basic setup and changed my startup.cfg to lowest graphics details code block, I've setup both fraps and OCAT to the same P key as my "Pause game key", loaded the mission and got pretty consistent results on my hardware from both tools.

I would be very interested in comparing average fps results across different configuration of HMD/CPU/GPU

Recommended GPU drivers
Nvidia: Latest drivers
AMD RX 400/500: 20.10.1
AMD 5000/6000: 22.10.2 (also latest at the moment of testing)

Test setup:

Steam VR 100%, 72hz, world scale 112%

Virtual Desktop, Streamer 1.24.1

Pico loaded Virtual Desktop 1.24.0

Results were very consistent across three runs, latest run results included.

fraps:

2022-11-04 15:25:32 - Il-2
Frames: 3344 - Time: 60000ms - Avg: 55.733 - Min: 48 - Max: 63

ocat:

File	Application Name	Compositor	Date and Time	Average FPS (Application)	Average frame time (ms) (Application)	95th-percentile frame time (ms) (Application)	99th-percentile frame time (ms) (Application)	99.9th-percentile frame time (ms) (Application)	Missed frames (Application)	Average number of missed frames (Application)	Maximum number of consecutive missed frames (Application)
OCAT-Il-2.exe-2022-11-04T152534.csv	Il-2.exe	DWM	20221104-152534	55.7	17.9	20	21.6	41	15	0	1

startup.cfg graphics setup for upper test:

[KEY = graphics]
	3dhud = 0
	adapter = 0
	bloom_enable = 0
	canopy_ref = 0
	desktop_center = 0
	detail_rt_res = 1024
	draw_distance = 0.27400
	far_blocks = 1
	fps_counter = 1
	fps_limit = 0
	full_height = 1080
	full_width = 1920
	fullscreen = 0
	gamma = 1.00000
	grass_distance = 0.00000
	hdr_enable = 0
	land_anisotropy = 0
	land_detail = 0
	land_tex_lods = 0
	landscape_mesh_quality = 1.00000
	max_cache_res = 1
	max_clouds_quality = 2
	mgpu_compatible = 0
	mirrors = 0
	msaa = 0
	multisampling = 0
	or_ca = 0.00000
	or_dummy = 0
	or_enable = 1
	or_height = 3120
	or_hud_rad = 0.40000
	or_hud_size = 0.60000
	or_ipd = 0.06305
	or_render_eye = 1
	or_sipdc = 0.06000
	or_width = 3120
	post_sharpen = 0
	preset = 2
	prop_blur_max_rpm_for_vr = 155
	rescale_target = 1.00000
	shadows_quality = 3
	ssao_enable = 0
	stereo_dof = 5.00000
	vsync = 0
	win_height = 1050
	win_width = 1680
[END]

Edited November 4, 2022 by 102nd-YU-cmirko

November 4, 2022

In the picture of the settings I can not see the settings for the clouds, what did you used? (I am not very familiar with using Startup.cfg files)

November 4, 2022

grass is off, clouds high

November 6, 2022

I have run the benchmark with the Pico4 exactly at the settings you indicated.

When Godlike resolution in VR is selected, then the 100% SteamVR resolution correspond to 3120x2130 as you indicated.

My system is a 5600X (standard settings), AIO liquid cooling CPU, 32MB RAM at 3600, latency 16-16-16-36, and a 3080 (not overclocked).

The fps reported by two runs with FRAPS:
Frames: 3229 - Time: 60000ms - Avg: 53.817 - Min: 46 - Max: 60
Frames: 3247 - Time: 60000ms - Avg: 54.117 - Min: 47 - Max: 60

The data reported by two other runs with OCAT:

Average FPS (Application)

Average frame time (ms) (Application)

95th-percentile frame time (ms) (Application)

99th-percentile frame time (ms) (Application)

99.9th-percentile frame time (ms) (Application)

Missed frames (Application)

Average number of missed frames (Application)

Maximum number of consecutive missed frames (Application)

53.5, 18.7, 21.2, 22.4, 44.5 0, 0.0, 0

54.4, 18.4, 20.9, 22.2, 42.7 0, 0.0, 0

Edited November 6, 2022 by chiliwili69

November 6, 2022

thanks!

November 7, 2022

I also did some more testing of the Pico4 but just changing the mode to Ultra, 2736x2736=15million pixels (I have a 3080 card, so Godlike mode is too much for it).

And also with the Index at 80Hz putting SteamVRSS at 166% (2596x2884=15million pixels)

Average FPS (Application)

Average frame time (ms) (Application)

95th-percentile frame time (ms) (Application)

99th-percentile frame time (ms) (Application)

99.9th-percentile frame time (ms) (Application)

Missed frames (Application)

Average number of missed frames (Application)

Maximum number of consecutive missed frames (Application)

PICO4

59.2, 16.9, 19.4, 20.6, 43.4, 0, 0.0, 0

58.3, 17.2, 19.5, 20.6, 42.7, 0, 0.0, 0

INDEX

65.5, 15.3, 18.0, 25.6, 42.1, 0, 0.0, 0

66.5, 15.0, 18.2, 25.5, 29.0, 0, 0.0, 0

November 7, 2022

your results with Index at 166% make total sense in comparison with number of pixels pushed across both devices on the same hardware. It seems though that AMD cards/drivers still have issues with il2 engine....

It would be very interesting to see somebody with 5800x3d/4090 run this test on their G2 or Pico4 - just to make an informed decision on possible upgrades this Christmas season

November 7, 2022

Nice one @chiliwili69 you might want to correct the load (fps) by adding the Hz into the equation as that's missing, by simply equalize results to 90Hz. All is per second, so easy:

Pico4 * 90/72

Index * 90 /80

For frametimes, turn it around.

They're both performing within normal variance imho. Neither one throttles, the performance isn't affected by the headsets. The Index scores a bit worse in the microstuttering - but that might well be random spikes and not tied to headset, which means:

Same performance.

Edited November 7, 2022 by SCG_Fenris_Wolf

November 10, 2022

On 11/4/2022 at 4:18 PM, 102nd-YU-cmirko said:

I would like to see relative performance on Pico4 with latest supported drivers and across GPU and CPU architectures - sort of a general test for VR users to see are there major discrepancies.

I tested my configuration x570, 5800x, 32gb DDR4 3800Mhz, 6900XT water cooled, latest windows10 22H2 19045.2193, latest AMD Chipset drivers, latest and VD recommended AMD Radeon drivers.

Pico loaded Virtual Desktop 1.24.0

Results were very consistent across three runs, latest run results included.

fraps:

2022-11-04 15:25:32 - Il-2
Frames: 3344 - Time: 60000ms - Avg: 55.733 - Min: 48 - Max: 63

I upgraded my CPU to 5800X3D, exactly same RAM profile as before, same bios as before

everything else is the same except new minor version of Virtual Desktop - 1.24.1

fraps results show 4fps gain

2022-11-10 11:22:31 - Il-2
Frames: 3565 - Time: 60000ms - Avg: 59.417 - Min: 52 - Max: 67

November 11, 2022

22 hours ago, 102nd-YU-cmirko said:

2022-11-10 11:22:31 - Il-2
Frames: 3565 - Time: 60000ms - Avg: 59.417 - Min: 52 - Max: 67

In order to know how much CPU/GPU you are constrained you can run the same test but with lower resolution, medium for example.

So if you still have around 59fps then it is a CPU bottleneck, but if you get 72fps then it means your GPU is the bottleneck. Just in this benhcmark.

November 11, 2022

I am GPU bottlenecked for sure at this moment (considering that I can not upgrade to a faster CPU in this socket :)) - I've played just one night online with new CPU, I gained around 4-8 fps average on Finish server (70+ players Kuban) which makes my upper online setting almost a perfect match for 72Hz refresh...., I'm considering playing with https://github.com/fholger/openvr_fsr to see results (Pico4 anyways has big distortion on edges of lens, so it could work very well with steamvr/VD combo)

November 12, 2022

just did another round of testing with tweaked CPU, my system is all core stable with FCLK 2000 and in 1:1 with RAM even with undervolt of -25mV (I followed the guide from this link on github.

Il2 doesn't like undervolting though - so with PBO2 tuner disabled (meaning default PBO curve from bios) and update in FCLK/RAM I gained another 1fps in average (I will not post results because this falls into test variance in my opinion)

I also tried https://github.com/fholger/openvr_fsr , after I finished CPU tweaking, with lower pasted configuration for my personal PICO4 experience (for me it feels like a good match with outer rim of the lens)

fraps results of test mission

Frames	Time (ms)	Min	Max	Avg
3802	60000	53	73	63.367

"fsr": {
    // enable image upscaling through AMD's FSR or NVIDIA's NIS
    "enabled": true,

    // if enabled, uses NVIDIA's Image Scaling instead of the default
    // AMD FidelityFX SuperResolution. Both algorithms work similarly, but produce
    // somewhat different results. You may want to experiment switching between the
    // two to determine which one you like better for a particular game.
    "useNIS": false,

    // Per-dimension render scale. If <1, will lower the game's render resolution
    // accordingly and afterwards upscale to the "native" resolution set in SteamVR.
    // If >1, the game will render at its "native" resolution, and afterwards the
    // image is upscaled to a higher resolution as per the given value.
    // If =1, effectively disables upsampling, but you'll still get the sharpening stage.
    // AMD presets:
    //   Ultra Quality => 0.77
    //   Quality       => 0.67
    //   Balanced      => 0.59
    //   Performance   => 0.50
    "renderScale": 0.77,

    // tune sharpness, values range from 0 to 1
    "sharpness": 0,
    
    // Only apply FSR/NIS to the given radius around the center of the image.
    // Anything outside this radius is upscaled by simple bilinear filtering,
    // which is cheaper and thus saves a bit of performance. Due to the design
    // of current HMD lenses, you can experiment with fairly small radii and may
    // still not see a noticeable difference.
    // Sensible values probably lie somewhere between [0.2, 1.0]. However, note
    // that, since the image is not spheric, even a value of 1.0 technically still
    // skips some pixels in the corner of the image, so if you want to completely
    // disable this optimization, you can choose a value of 2.
    // IMPORTANT: if you face issues like the view appearing offset or mismatched
    // between the eyes, turn this optimization off by setting the value to 2.0
    "radius": 0.8,

    // if enabled, applies a negative LOD bias to texture MIP levels
    // should theoretically improve texture detail in the upscaled image
    // IMPORTANT: if you experience issues with rendering like disappearing
    // textures or strange patterns in the rendering, try turning this off
    // by setting the value to false.
    "applyMIPBias": true,
    
    // If enabled, will visualize the radius to which FSR/NIS is applied.
    // Will also periodically log the GPU cost for applying FSR/NIS in the
    // current configuration.
    "debugMode": false,

November 12, 2022

16 minutes ago, 102nd-YU-cmirko said:

Il2 doesn't like undervolting though

Not true. My biggest gains in performance and efficiency with a 5900X came by tuning PBO and the power limits. Those two go hand in hand, you have to tune PBO based on the limits you set otherwise it will be unstable or simply show no benefit.

You have to aim for the lowest power usage you can get while using the highest negative shift your CPU's preferred cores allow for. This reduces heat and allows the latter to boost higher for longer.

November 12, 2022

I may have not expressed myself correctly there - what you described was my experience with my previous 5800x (which I undervolted with HYDRA values through BIOS) on the other hand 5800X3D PBO can not be changed except through the PBO2 software after windows starts, whichever negative value I apply there lowers my score in upper described testing compared to clean windows start, if I apply -30mv to all cores prior to benchmark start I get results like with my previous 5800x with ram and fclk on 1900Mhz

bear in mind, my cpu is completely stable with -30mv on all cores in over six hours of Prime95 stress testing (fclk 2000 and ddr4 4000 cl16)

I'll do some more testing with positive value offset for windows preferred cores when I find the time....

November 20, 2022

I'm not sure these tests are accurate as I've noticed something weird in my mine. When looking at the frame time it's showing that I should be hitting a stable 90 FPS no problem, yet my real FPS is dropping to the 70s. Compared to my Reverb G2 I'm seeing the same frame timing and I'm getting a stable 90 FPS on the same mission, resolution, settings, etc. My theory is these testing methods are only showing what happens before the image goes through the compression process, and there's actually a very significant performance loss in the final output that doesn't show in things like FPSVR or others.

November 21, 2022

23 hours ago, =MERCS=JenkemJunkie said:

My theory is these testing methods are only showing what happens before the image goes through the compression process

This is interesting. For VR devices based on DP cable all tools (Fraps, fpsVR, SteamVR framerates, etc) reports exactly what it is sent to the DP cable which is exactly what it is seen at the device.

For VR devices based on the compression methods (either wifi or USB cable with no DP support) then it seems that we need to analyze what they really report.

According to your image, I see a solid 9.5ms framerate for GPU (in fpsVR overlay and SteamVR overlay) so the fps should be then solid 90fps, but fpsVR and Pico4 overlays report 75fps.

So it seems that fpsVR and Pico4 report well the true fps. And perhaps fraps or other tools are reporting 90fps.

Edited November 21, 2022 by chiliwili69

November 21, 2022

I got an answer on the VD discord. Looks like there's an issue they're working on causing too large of a performance hit, so hopefully this gets fixed later. Also they confirmed FPSVR is showing the frame time pre-compression, the 75 FPS is caused by the performance hit post-compression.

November 21, 2022

it's slightly strange, this difference of experience with nvidia and amd in combination with pico4, both you and chiliwili have (had) stutter issues.... and I was asking ggodin on VD discord if he can provide bigger encoding resolution than godlike because I felt that pico4 runs much better than my old HP G2 (at exact settings) and I don't like the SS effect personally.

November 21, 2022

Seems weird that NVidia would be having problems and AMD is the chosen one for once in VR. Have you tried running tests in a situation that is a joke on the CPU like the quick mission solo free flights? If you crank up the graphics you can see if your GPU frame time in something like FPSVR matches your actual FPS.

November 21, 2022

10 hours ago, =MERCS=JenkemJunkie said:

Also they confirmed FPSVR is showing the frame time pre-compression, the 75 FPS is caused by the performance hit post-compression.

So, fpsVR report frametimes of the pre-compression but when showing FPS they are reporting FPS shown in the headset (adding GPU encoding + network + XRchip decoding), right?

November 21, 2022

15 minutes ago, chiliwili69 said:

So, fpsVR report frametimes of the pre-compression but when showing FPS they are reporting FPS shown in the headset (adding GPU encoding + network + XRchip decoding), right?

Yeah if you divide 1000 by the frame time you'd get the FPS you would be getting if you were connected via display port. The FPS is showing your real FPS after all the compression bullshit. So I should be getting 1000/9.6 = ~104 FPS but instead compression is knocking me down to 75. That's a huge hit just for going wireless in a seated sim.

November 21, 2022

have you tried lowering the bitrate to under 150 mbits ?

November 21, 2022

52 minutes ago, 102nd-YU-cmirko said:

have you tried lowering the bitrate to under 150 mbits ?

Yeah, theres always a big performance hit.

November 21, 2022

we just tested new pico4 updated to 5.2.2 on friends x570/5800x/32gbPC3600/3080ti latest nvidia drivers - his game runs well on 135mbits/godlike

also this - https://discord.com/channels/564087419918483486/1027563564480864356/1044390817827131503

Edited November 21, 2022 by 102nd-YU-cmirko

November 22, 2022

I'm getting the same results post 5.2.2 at 135 bitrate, but it's good it's being worked on.

November 22, 2022

When I was testing the Pico4 before I return it, I always was using the 72Hz mode.

The reason is that according to ggodin (look here) for the same bitrate (ie 150Mbps HVEC) the 72Hz mode will provide better image quality since less frames need to be encoded+transmitted+decoded for the same bitrate.

In addition to that, the 72Hz is less demanding CPU wise (important for complex scenarios with lot of objects around) and GPU wise (Godlike resolution or additional %SS).

I didn´t notice any difference from 72Hz to 90Hz. Normally I always has played at 80Hz for the Index.

November 22, 2022

Yeah, I plan to ultimately run 72 hz, I was using 90 for easier testing. The performance loss will always be there regardless of whatever combo of settings you use. So instead of running say Godlike / x2 MSAA at 72 hz, I'll be stuck at High / x0 MSAA at 72 hz for example. On the bright side supposedly there's a patch coming to close the performance gap at least somewhat.

November 22, 2022

maybe make a test with settings as in first post ?

would be interesting to see your relative performance on 72Hz (also in relation to chili's results)

November 22, 2022

I can run it after work later. Is it a CPU heavy test mission with lots of objects and AI? I noticed those tests don't separate CPU and GPU frame time, so if the mission is CPU bottlenecked it wont be a good test for testing GPU performance loss, but I'll see what's going on in the test when I run it.

November 22, 2022

I'm not really interested in the pico encoding/decoding performance loss as much as a relative game performance at 72Hz, thanks for your time ^^

November 22, 2022

Hz wont matter in the posts people have posted because no one had broken 72 FPS. If your getting 50 FPS on 72hz, you'll get 50 FPS on 90 hz. The decoding loss is what matters because it means 30% of your expensive graphics card is being wasted, and you could be running better graphics settings for the same FPS if that gap is closed. If your not bothered by lower settings, then more tests wont provide new info, just run low graphics settings and have fun.

November 22, 2022

20 hours ago, =MERCS=JenkemJunkie said:

Yeah if you divide 1000 by the frame time you'd get the FPS you would be getting if you were connected via display port. The FPS is showing your real FPS after all the compression bullshit. So I should be getting 1000/9.6 = ~104 FPS but instead compression is knocking me down to 75. That's a huge hit just for going wireless in a seated sim.

Thats a huge fps loss if i am getting it right due to compression!!! To be honest thats the same thing i discovered going from Q2 to G2.

November 22, 2022

Feels bad ?. Q2 to G2 lost you FPS? I woumd have thought the uncompressed G2 would have done better, but I've never had a quest.

November 22, 2022

23 minutes ago, =MERCS=JenkemJunkie said:

Feels bad ?. Q2 to G2 lost you FPS? I woumd have thought the uncompressed G2 would have done better, but I've never had a quest.

Sorry i wasn't probably clear... i mean i had fps gain going from Q2 to G2, meaning G2 works better for me.

November 22, 2022

That makes more sense. Yeah compression is much worse then I thought it would be. I'll have to look into figuring out how hard it would be to solder a display port connector onto my Pico. What could go wrong with opening up expensive things, and doing things I half understand?

December 9, 2022

On 11/10/2022 at 11:30 AM, 102nd-YU-cmirko said:

I upgraded my CPU to 5800X3D, exactly same RAM profile as before, same bios as before

everything else is the same except new minor version of Virtual Desktop - 1.24.1

fraps results show 4fps gain

2022-11-10 11:22:31 - Il-2
Frames: 3565 - Time: 60000ms - Avg: 59.417 - Min: 52 - Max: 67

after Pico4 update to 5.2.7

VD streamer 1.25.7

VD installed in Pico4 1.25.6

latest radeon (recommended bv VD) and latest AMD mbo drivers

2022-12-09 14:27:10 - Il-2
Frames: 4170 - Time: 60000ms - Avg: 69.500 - Min: 62 - Max: 73

December 9, 2022

opencomposite results

exact setup like upper il2 tests in startup.cfg

2022-12-09 14:43:32 - Il-2

Frames: 4179 - Time: 60000ms - Avg: 69.650 - Min: 62 - Max: 73

both tests with these opencomposite settings

my new online setup results (lower block for reference)

2022-12-09 14:29:22 - Il-2
Frames: 4137 - Time: 60000ms - Avg: 68.950 - Min: 61 - Max: 73

update of graphics block and latest tests with 1.25.10 streamer and in HMD

2022-12-21 13:24:47 - Il-2
Frames: 4127 - Time: 60000ms - Avg: 68.783 - Min: 61 - Max: 73

[KEY = graphics]
	3dhud = 0
	adapter = 0
	bloom_enable = 0
	canopy_ref = 0
	desktop_center = 0
	detail_rt_res = 4096
	draw_distance = 0.54500
	far_blocks = 1
	fps_counter = 1
	fps_limit = 0
	full_height = 1080
	full_width = 1920
	fullscreen = 0
	gamma = 1.00000
	grass_distance = 0.00000
	hdr_enable = 0
	land_anisotropy = 16
	land_detail = 7
	land_tex_lods = 3
	landscape_mesh_quality = 1.00000
	max_cache_res = 1
	max_clouds_quality = 2
	mgpu_compatible = 0
	mirrors = 2
	msaa = 1
	multisampling = 1
	or_ca = 0.00000
	or_dummy = 0
	or_enable = 1
	or_height = 3120
	or_hud_rad = 0.55000
	or_hud_size = 0.75000
	or_ipd = 0.06540
	or_render_eye = 0
	or_sipdc = 0.00000
	or_width = 3120
	post_sharpen = 0
	preset = 1
	prop_blur_max_rpm_for_vr = 155
	rescale_target = 1.00000
	shadows_quality = 3
	ssao_enable = 0
	stereo_dof = 5.00000
	vsync = 0
	win_height = 1080
	win_width = 1920
[END]

Edited December 21, 2022 by 102nd-YU-cmirko

December 14, 2022

I tested my Pico after the update and got much closer to hitting what my frame time says I should be getting this time. My frame time of 13.4 ms means I should be hitting ~74 FPS and I'm getting 68, so that's not bad considering it's wireless and going through the compression process. My G2 was hitting 71-72 in the same scene with the same settings. The extra blur/artifacts from compression are still noticeable though, so the G2 still looks better with the same settings, but the performance difference isn't such a big deal any more. Stutters seem to have improved, but they're still there, and is likely a Pico issue not related to VD. The framerate will still wander a few FPS around 72 or 90 even when using Potato settings, and the tracking still seems to skip frames causing very annoying jumps in the picture.

December 21, 2022

tested latest streamer and update, no change in performance on the original dll, but with some more testing, I found out I can easily increase my online settings ^^

January 13, 2023

Pico4 update just dropped for me

Change log:

Added app permission management to protect users' privacy;
Optimized the color seethrough algorithm to reduce visual distortions;
Optimized the display of status prompts during screencast;
Supported triggering Play Boundary with hand tracking. Added effects for wall holes and half wall to Stationary Boundary. Shortened the process for setting Stationary Boundary;
Updated the wired connection protocol for streaming. Please use the latest version of the Streaming Assistant for PC for better streaming effects;
Added Parental Control mode to help guide teenagers to use the VR device healthily;
Optimized system performance and stability.

I use virtual desktop as before and performance is virtually the same (neither VD streamer nor my gpu drivers have changed)

2023-01-13 12:51:04 - Il-2
Frames: 4157 - Time: 60000ms - Avg: 69.283 - Min: 62 - Max: 73

Pico4 performance testing

Recommended Posts

102nd-YU-cmirko

chiliwili69

102nd-YU-cmirko

chiliwili69

102nd-YU-cmirko

chiliwili69

102nd-YU-cmirko

SCG_Fenris_Wolf

102nd-YU-cmirko

chiliwili69

102nd-YU-cmirko

102nd-YU-cmirko

firdimigdi

102nd-YU-cmirko

=MERCS=JenkemJunkie

chiliwili69

=MERCS=JenkemJunkie

102nd-YU-cmirko

=MERCS=JenkemJunkie

chiliwili69

=MERCS=JenkemJunkie

102nd-YU-cmirko

=MERCS=JenkemJunkie

102nd-YU-cmirko

=MERCS=JenkemJunkie

chiliwili69

=MERCS=JenkemJunkie

102nd-YU-cmirko

=MERCS=JenkemJunkie

102nd-YU-cmirko

=MERCS=JenkemJunkie

dgiatr

=MERCS=JenkemJunkie

dgiatr

=MERCS=JenkemJunkie

102nd-YU-cmirko

102nd-YU-cmirko

=MERCS=JenkemJunkie

102nd-YU-cmirko

102nd-YU-cmirko

Create an account or sign in to comment

Create an account

Sign in

Browse All

Activity