Hey, welcome to our WiFi 6 deep dive & real-world testing blog.
We’ve just hosted a WiFi 6 Network Nomads event with Natilik and put quite a lot of time and effort into preparing for this event, recorded a juicy podcast on WiFi 6 with David Coleman, studied hard & tested ax with some WiFi 6 clients on a WiFi 6 AP running beta code enabling some ax features 😉
We both wanted to share with you our real-world findings and experience of WiFi 6.
Before we jump into our testing and finding lets first have a quick recap on the evolution of WiFi:
Just wanted to mention key milestones here:
Now that we have recapped on the evolution of WiFi – let’s look at 802.11ax at a high-level overview of what’s new and improved:
We know everyone is dying to see some techie stuff here, so there we go. Here comes our OneNote notes (forgive us not converting it to book style / essay / marketing leaflet formatting hehe). This is a mixture of knowledge gathered by listening to other ax podcasts (thanks CTS!), reading ax blogs (thanks David Coleman!) and our real world testing.
Easy, right? Let’s take a look at a simple diagram visualising main OFDMA concepts:
To make it even easier, this is how subcarriers look like. Not sure where Francois and Rowell at CTS have found it, but this is the best OFDMA Subcarriers structure on 20MHz channel we’ve seen so far:
Here are the diagrams extracted from the 802.11ax draft document detailing the structure of the subcarriers for each channel width using different RUs sizes:
Lastly, please see the full MCS table below. Take some time to digest it. It took us a moment to get it 😉
Now, let’s switch our focus to BSS Colouring 🙂
TWT is next on the list 🙂
In short: MU-MIMO is used to allow multiple simultaneous AP <-> STA conversations on a single AP. Sounds great, but there are some conditions that must be met for MU-MIMO to work:
When you think more about it, it adds even more complexity – we need more antennas on the AP for MU-MIMO to make sense. 4×4:3 (quite popular mix on the modern APs) would allow us to use 2×2:2 and 2×2:1 for example. Some vendors start packing the APs with 8×8 and it’s great for MU-MIMO, but how about AP power consumption? More antennas or more radios = increased power consumption, on both AP and the client side. 802.3at (30W) might no longer cut it and we’re not sure that having more antennas is worth upgrading switching infrastructure to support UPoE (Cisco, 60W) or 802.3bt (standard, 90W). Additionally, we are not aware of any clients supporting MU-MIMO in both directions.
Lastly, let’s take a look at the new modulation scheme!
There is popular theory that 1024 QAM is a waste of time, as you need to literally place your device on the AP to achieve it. Is it true? Not necessarily! We’ve run some test (see ‘Demo’ later on in this blog) and maintained MCS 10 and 11 (both using 1024 QAM) while moving quite far away from the AP. Let’s come back to that in a sec.
Now, what is 1024 QAM and how would it change our lives? It’s just a faster modulation scheme. Iteration, not revolution, offering up to 20% gain in theoretical throughout (less in the real life). It’s still good to have. More throughout = less time spent using the airtime.
It makes even more sense when used with OFDMA and RUs – narrower ‘baby channels’ (RUs) would offer higher SNR than 20MHz OFDMA 242-tonnes or 20MHz OFDM, and therefore it would be easier to maintain 1024 QAM over even longer distances.
Let’s move to the APs. We wanted to cover and highlight some of the pre-standard APs that have been released and explain some of the wording + terminology used.
Cisco WiFi 6 WAPs:
Meraki WiFi 6 WAPs
Like us you might have been wondering what is the difference between certifiable and compatible? We reached out to our contacts at Cisco and got the following responses:
“There are some Wi-Fi 6 access points already on the market, targeted for early adopters and customers who are eager to test the new standard. The access points that are released early will be pre-standard APs because the standard will not yet have been ratified. This means key features that are part of Wi-Fi 6 may not be supported on some of these initial, pre-standard access points. However, when available, some of these access points will be able to become certified through software updates and Wi-Fi 6 features will be supported. This approach is similar to the introduction of prior generations such 802.11ac and 802.11n.
first iteration of 8×8 in the 9117 (incidentally the same as all other manufacturers with the same chipset) does not support OFDMA in UL, therefore, we are saying that this AP will be compatible with wifi6 from the WiFi alliance perspective – future versions of 8×8 APs will be certifiable. With the 9115 and 9120 we are confident that there will be no changes to the standard for those AP’s, so confident that they will be certifiable to WiFi Alliance WiFi 6″
A few things we want to make me clear here:
Certifiable = will be WiFi 6 compliant in the future with a software update
Compatible = follows draft but will not support all WiFi 6 features
OFDMA is a new WiFi 6 thing and it’s mandatory in both directions
No OFDMA = no WiFi 6 compliance
Ok we feel like we have recapped the evolution of WiFi, what’s new and improved in WiFi 6 and the difference in some of the WiFi 6 WAPs. Let’s move on to our testing & findings.
We finally got our hands on a WiFi 6 AP (Cisco Cat 9115 – thank you Cisco!) and 2 x WiFi 6 devices (Samsung s10e) and it was safe to say we were excited as they love anything to do with WiFi 🙂
That’s what we used 🙂 Cisco 9115 AP & 2x Samsung S10e:
A very happy Mac & Matt, featuring 2 x Ekahau Sidekicks used for Spectrum Analysis and Packet Captures:
In this set up at Mac’s home productions network he currently has running a Cisco WLC3504 which was upgraded to AireOS 8.9 as this is the first version of software that supports WiFi 6 WAPs.
Cisco WLC 3504 on Cisco AireOS 8.9 code, 802.11ax configuration 1:
Cisco WLC 3504 802.11ax configuration 2:
Now that everything was configured correctly, we connected the two Samsung S10e’s to the Cisco 9115 AP and the little 6 logo now appeared next to the WiFi icon which we both thought was pretty cool and exciting! Mac couldn’t sleep for a week because of this over-excitement.
We decided to look at some wireless packet captures to see what was going on.
As we can see above there is no support for UL & DL OFDMA, BSS Colouring, UL & DL MU-MIMO, 1024-QAM and TWT, meaning that no WiFi 6 features are supported on Cisco AP C9115 running 8.9 code!
We then moved to check our client’s ax capabilities 🙂 We’ll be looking at probe request, as this gives us a clearer pic of what the client is really capable of. Looking at, in example, authentication or association request would show us client’s ‘response’ to the capabilities presented by the AP and client would most likely want to match them in its responses. So even if a client device supports more ax features, we probably wouldn’t see that in captures.
So, from what we could see here was that all the new features of WiFi 6 and what would make a wireless device WiFi 6, seemed to be not supported on neither the AP or either of the phones!
Now we take a look at the spectrum analysis of Xiaomi WiFi 5 device connect to WiFi 6 enabled wireless network connected to WiFi 6 Cisco Cat 9115 AP and run a nPerf speed test.
Xiaomi WiFi 5 phone:
Samsung WiFi 6 s10e:
We can clearly see both WiFi 5 & 6 devices associated with an ax AP use OFDM.
Bit confused like us that the WiFi 6 device looks to have a very similar spectrum pattern to the WiFi 5 device?
We decided to compare the beacon frame of some other vendor WAPs to see if anyone else was supporting any WiFi 6 Features yet:
We can see here the AP is using draft 3.0 and BSS Coloring is enabled as here says disabled: false
We can see here the AP is using draft 3.0 and BSS Coloring is enabled as here says disabled: false
Quick recap of what we’ve seen in above captures:
We now wanted to cry (hehe) and we’ve reached out to Cisco with our findings. Cisco has confirmed that 8.9 AirOS code just provides support for the WiFi 6 WAP’s to join the WLC but no WiFi6 features – so they kindly added us to their Beta testing programme and gave us a copy of 8.10 which would turn on a couple of features of WiFi 6. Here is what’s now supported:
We’ve installed the Beta 8.10 Code on Mac’s production WLC3504 to his wife’s dissatisfaction and began testing again:
Checked to make sure what the Cisco Cat 9115 AP was supporting now – the below shows just loggs from the AP SSH showing that our secret features (most importantly OFDMA in both directions) should now be supported:
We’ve preformed some more wireless PCAPs, so now what features do we support that was not here before?
We have highlighted in blue everything that is still not supported from the previous screens shots in AireOS 8.9 and everything in red is what is now supported in AireOS 8.10.
Operating Mode (OM) Control Field is now supported! It allows STA to suspend participation for synchronized UL-OFDMA and contend for the medium for an independent uplink transmission.
1024 QAM for 242-tone RU (full 20MHz channel width) is also supported now! Our hopes are getting higher and higher 🙂
And that’s it! Let’s quickly recap what should theoretically be supported, post upgrading our 8.9 AireOS to the new, cutting edge, breathtaking beta code version 8.10 🙂
MCS0-11 and OFDMA UL & DL are now supported on both Cisco C9115 AP and Samsung S10. Both devices use 802.11ax Draft 3.0 as a base. It’s looking promising on paper now 🙂
We re-ran the same speed test and analysed the spectrum and compared the results of the Samsung S10e’s WiFi 6 vs Mac’s Xiaomi WiFi 5 device. Testing methodology didn’t change – we’ve associated one device at a time, there was completely no spectrum activity on our test channel 36, no neighbours, no interferences, very stable and low noise floor. This is what we’ve seen:
Xiaomi WiFi 5 phone (WLC running 8.10 beta):
No change here (as expected), still hitting 83%-ish in spectrum utilisation, using OFDM over 20MHz channel.
Samsung WiFi 6 s10e Test 1 (WLC running 8.10 beta):
Samsung WiFi 6 s10e Test 2 (WLC running 8.10 beta):
What’s happening on channel 36 here when running the speed tests on the Samsung s10!?
Look at Test 1. We can see that the channel is being split in half – clearly some secret sauce functionality of WiFi 6 is happening!
We’ve run multiple tests to make sure we’re not dreaming. On ‘Test 2’, client decides to use entire channel. But is OFDMA still in use? It appears it is! When you watch closely, you’ll see several ‘peaks’ inside channel 36. Those peaks look like 26-tones RUs. We are almost sure OFDMA is in use throughout both tests! Please note that in both tests spectrum utilisation peaked at around 70%.
There is one more question. Or even two 🙂 Why would the same device decide to use half of the channel in one test, and entire channel in the other? Unfortunately, even after chatting about it with our Jedi Master, Peter MacKenzie, we didn’t get to a definite conclusion. Our best educated guess is that the device has decided (most likely using OFDMA Random Access – where the decision of RU allocation comes from a client) to use half of the channel to improve SNR and achieve MCS 11. Again, it’s just a guess. Let us know if you have better ideas about what happened there! 🙂
Second question would be around the ax client (S10) utilising less spectrum than ac client (Xiaomi) during exactly same test. We know that our test, where we just looked at a spectrum utilisation, is not too scientific as the speedtest was capped by Mac’s Internet pipe (50Mbps) and we should really use local iperf server ideally to gauge real throughout gain in ax vs ac. We’ve run multiple tests (more than 10) throughout the day and the results were consistent – ax client (S10) was utilising approximately 15% spectrum than ac client (Xiaomi) while running nPerf.
Why would we see this improvement you ask? Great question!
Since OFDM (ac) and OFDMA (ax) really make no difference in terms of throughput for a single client, we suspected that our WiFi 5 and WiFi 6 devices were operating using different data rates.
We’ve picked up any random data packet transmitted by WiFi 5 device and realised it’s operating using MCS8, 2SS, 20MHz channel, ac and short guard interval. Data rate used was 173.3 Mbps and this is as fast as 2 Spatial Stream 802.11ac device can go over 20MHz channel.
Sweet, we now wanted to check the same on the WiFi 6 data captures. Wait, what data captures? Silly Ninjas. We’ve only got a 802.11ac capable packets capturing device (Ekahau Sidekick) and while we can see control frames coming from and to our beefy S10 and they’re send with a lowest mandatory BSS rates (in our test we’ve used 12Mbps mandatory, all lower disable, all higher supported), we won’t be able to see any data frames. Not a single one. None. Nada. But we know they’re there! Look at delta times after S10 has got a green light from the AP to send data (Clear-to-Send) – there is clearly something missing.
Since we couldn’t validate S10 data rates in captures, we had to rely on what the device itself was reporting back 🙂 Again, not the most scientific test but we must go with what we have 😉
We’ve used this opportunity to also check the max distance from an AP with Line-of-Sight to the S10. We’ve place the AP on the tripod, grabbed a laser tool and started moving away from the AP. With entire 20MHz used, we could maintain MCS11 (QAM 1024 5/6) over the first 6 metres. We would then drop to MCS10 (QAM 1024 3/4) when 7-10 away from the AP. We went as far from the AP as the garden allowed, and reached the wall (literally) being good 13 metres away from the AP, at which point we dropped to MCS 9 (QAM 256 5/6), which is still quite sweet.
Max data rate achieved by S10 (286Mbps) was considerably higher than the data rate used by Xiaomi (173Mbps) and this is why we’ve seen lower spectrum utilisation while downloading 40MB file as part of our test.
See the MCS table below. We can see that both WiFi 5 and WiFi 6 test clients were operating at their highest achievable MCS rates. Our S10 devices were happily reaching MCS11, that is an ax rate. It all makes sense now. S10s were using either ax OFDM, 1024-QAM 5/6 with SGI at 2SS or ax OFDMA 24-tone RU 1024-QAM 5/6 SGI at 2SS. Either way, it’s impressive.
To summarise, most of the ax features are still not entirely supported on both client and AP side but we can clearly see that software updates are bringing more and more WiFi 6 improvements. Within the last 2 weeks alone we’ve seen a juicy update from Cisco (8.10 beta) and it should be soon followed by a publicly available 8.10 AireOS version adding support for even more ax features like TWT and BSS Colouring (at which point Cisco should support all WiFi 6 enhancements!). We’ve had a Samsung S10 system update right after taking that phone out of the box. A week later (in between our 8.9 and 8.10 tests) we’ve had another S10 system update ready. Both listing ‘WiFi Improvement’ as the main feature in the changelog. Intel and the others started promoting their mobile WiFi 6 chipsets, that should find their way to consumer laptops later this year. See? 802.11ax aka WiFi 6 is indeed just behind the corner.
Samsung has started a good trend here packing ax radios into S10 line. AP vendors are super brave trying to implement new WiFi iteration before the standard is even ratified. And you know what? We should probably give those vendors a little credit. It must be challenging and expensive and without their vision and drive for innovation, we wouldn’t be adopting those new toys and big boys’ gadgets as quickly as we’re adopting them now.
Question we have been asked recently is would we upgrade to WiFi 6 APs right now – and our answer after all of the studying and real world testing is that probably yes – why not 😉 At first you may just be getting a glorified WiFi 5 AP but it is still going to be a pretty good god damn WiFi 5 AP and when them software updates start coming and more support for the WiFi 6 features along with more WiFi 6 clients, we should see benefits all across the board.
11 Responses
Awesome job fellas! Very extensive testing for 802.11ax devices even though everything is early in the game.
Thank you for the amazing feedback David 🙂
Thats a great start to proceedings guys, i think i mentioned previously, and coming directly from a WiFi vendor in Beta test for 802.11ax myself, the entire roadmap from both Broadcom and Qualcom are driving the progress that the developers can work with. The chipset SDK’s release pretty much ties up every WiFi vendor in the same way. I was not suprised to find that most features were not there yet, its the same story from my testing too. So far just 1024 QAM and OFDMA. The rest will come, but pre-ratification it can be a bit of a leap of faith. It will be awesome when we get it all though.
Hey Colin, completely agree – even with just a few of the new features turned on we found it pretty cool to see OFDMA working and excited to see more features in the near future 🙂
Have you also contacted EnGenius and asked them about a beta release for their EWS357AP access point or a roadmap for 802.11ax OFDMA features?
Not sure if you know about what the folks behind the WLANPi figured out about client capabilities or not. They discovered that clients wouldn’t advertise their true capabilities until an AP advertises it can support it. They created a fake AP on the WLANPi to get clients to reveal their true capabilities. If you aren’t aware of it, you should talk to them, it’s pretty interesting.
Hey Jim, the WLANPi does seem like a nice piece of kit but we are yet to have our hands on one. What you are describing is why we used the decode from the probe request of the Samsung as we knew that sometimes clients do not advertise full capabilities and/ or they will sometimes match the capabilities of the AP that it is connecting to! Good tip though – thanks!
You guys rock. Good piece, solid captures, not a bad choice for analysis device either 😉 We saw some of the same OFDMA-seeming spectrum funkiness in March , which was a bit odd. https://twitter.com/jussikiviniemi/status/1105908569461932034?lang=en
Thank you Jussi the Janitor 😉 We’ve seen it back in March, was a little bit unexpected but still very cool! Almost as cool as your nickname here and on CLUS badge lol
Greetings from Russia, guys.
Such an amazing test, thank you so much for your work! I have one simple question. Did you test QAM1024 with two clients on 106-tone RUs? Of course, it should be easier for client to get to better modulation due to narrower RUs, but test is always better than educated guess 🙂
Hello,
Thanks for the great work.. i have a few comments. In a section on 802.11ax OFDMA, you have made comparisons between OFDMA and OFDM.. i wanted to add a few points. I presume that you are comparing the OFDM of 11a/g/n/ac with the OFDMA of 11ax. Please note that 11ax OFDMA is also built on top of OFDM definition which is modified compared to earlier standards. In 11ax, we can have a plain OFDM transmission called as HE-SU-PPDU which can utilize the advantages of narrower subcarrier spacing, different GIs, and the higher order modulation. In fact the absolute peak rate of 11ax which is around 9.6 Gbps can only be obtained with a pure HE-SU-PPDU as we will have best resource utilization there. You have captured the benefits of OFDMA very well.
Another point is regarding the comment on Cisco 9115.. u have mentioned that it does not support OFDMA based on the capabilities analysis in the beacon. Could you please point out how you were able to conclude based on the beacon. Thanks once again.