Mac Deryng

WN Video 008 – Configure Juniper Switch Using Mist Dashboard

Welcome to the WiFi Ninjas Video where we domonstrate how to configure a Juniper Switch using Mist Dashboard. Curious about pros, cons, limitations and what’s available for us to tinker with in this recent Juniper Mist wired integration? Dig in! We tried to cover it all 😉 Enjoy!

WN Blog 030 – USB Network Adapter on VMware ESXi (tested on NUC 10)

Welcome to our latest blog!

If you’re like me and want to have most compact, cold, silent and energy efficient ESXi server at home, you probably looked at Intel NUC more than once 😉

I’m now upgrading (or rather downgrading?) my lovely 2U ESXi that built myself 2 years ago. It is still powerful, but I have no space for it. I will tell you why.

My server room is located under the stairs. It was filled with equipment – Cisco FTD 5506x firewall, Cisco 3560cx switch, Cisco WLC3504, Cisco 9120, 9130, 3x 3702 APs, 2U ESXi server and UPS battery ensuring my NFS storage is not corrupted in the event of a power cut.

All that boxes generated so much heat, that I have decided to make space for clothes airer – my washing was dry just 2-3 hours after putting it inside the server room 🙂 This space could also be used as sauna. 

I then decided to simplify my home network and lab. Moved NAS to the cloud, virtualised firewall (chosen Untangle – amazingly sleek), mounted APs properly outside of the server room and switched to Mist.

Now I’m down to just an ESXi server, 1 small, passive, lovely Juniper EX2300-C-12P switch (that can be managed from Mist cloud!) and 2x Mist APs – AP41 and AP43. No more controllers, batteries, firewalls.

Do you see what my problem is?

Washing don’t dry out quickly anymore, as it’s cold in the server room!

To fix this, I will put a dehumidifier in that room. But I need more space to do it. This is why I wanted to switch to micro server (NUC) – so I can replace the rack with massive ESXi server with dehumidifier.

I ordered 10th Gen Intel NUC with 64GB of RAM and 2TB SSD – it’s more than enough to run my production and lab networks. But since I no longer have a physical firewall, I encountered a challenge – NUC has only one wired NIC card. I now needed two – one leg connected to BT fibre converter (WAN PPPoE) and one leg connected to the switch (LAN). 

After putting ESXi 7.0 on my NUC (it was quite challenging – NUC Intel wired NIC is not supported by ESXi and it required adding Intel NIC drivers to the ESXi image – thank you Bernhard @WiFi_Burns for your help!), I realised that my USB NICs are not recognised when connected.

Additional drivers are required to make it work.

Since we WiFi nerds don’t use VMware excessively, I personally found instructions available from VMware quite confusing and difficult to follow, so I’ve decided to put all steps needed to make external USB network adapters work on Intel NUC running ESXi.

Steps required to use USB Network Adapter on VMware ESXi (tested on NUC 10)

1. Download .zip file from here, making sure you get the correct version for your ESXi (currently 6.5, 6.7 or 7.0 are supported)

Image

2. Make sure Safari (assuming you’re using Safari) doesn’t automatically unzip the downloaded file (Safari > Preferences > General > untick ‘Open “safe” files after downloading’)

General 
Tabs 
General 
AutoFill Passwords Search Security Privacy Websites Extensions Advanced 
Safari opens with: 
New windows open with: 
New tabs open with: 
Homepage: 
Remove history items: 
Favourites shows: 
Top Sites shows: 
windows from last session 
Favourites 
Favourites 
http://google.co.uk/ 
Set to Current Page 
one year 
Favourites 
24 sites

3. Upload .zip to ESXi datastore

vrnvvare 
Navigator 
Host 
Manage 
Monitor 
ESXi 
WN-ESXi7.O - Storage 
Datastores Adapters 
New datastore 
Name 
datastorel 
Devices 
Persistent Memory 
Register a VM Datastore browser 
Virtual Machines 
• Storage 
¯ datastorel 
Monitor 
More 
Drive Type 
Capacity 
Provisioned 
Create directory 
roota I ( 
Free 
I Refresh 
Download 
Upload 
datastorel 
Delete Move Copy 
.sdd.sf

4. Enable SSH (Web Client > Host > Manage > Services > SSH > Start)

vmware• ESXi- 
O The service TSM-SSH was successfully started 
- dismiss 
Hardware 
Hos 
Monitor 
5.] Virtual Machines 
Storage 
datastorel 
Monitor 
More storagem 
Networking 
System 
Sta 
Name 
attestd 
DCUI 
lbtd 
ntpd 
Ljcensjng 
Packages 
ces 
Stop Restart I 
e Refresh 
I Actions 
Description 
attestd 
Direct Console UI 
Load-Based Teaming Daemon 
Active Directory Service 
NTP Daemon 
PC,'SC Smart Card Daemon 
PTP Daemon 
CIM Server 
SNMP Server 
ESXi Shell 
v 
sfcbd-watchdog 
snmpd 
Security & users 
Status 
Stopped 
Running 
Stopped 
Running 
Stopped 
Stopped 
Stopped 
Stopped 
Stopped 
Stopped 
Source 
Base system 
Base system 
Base system 
Base system 
Base system 
Base system 
Base system 
Base system 
Base system 
Base system

5. SSH to the ESXi terminal with your favourite client or straight from the Web Client (Host > Actions > SSH Console)

vmware 
Navigator 
Host 
Monitor 
ESXü 
WN-ESXi7.O 
6) Get vCenterS«ver 1 CreateJRegisterVM I o Shutdown • Reboot I Refresh 
WN-ESXi7.O 
Version: 
V'•tual Macthes 
Storage 
datastorel 
Monitor 
More storage„. 
Networking 
7.0.0 (Build 15843807) 
Maintenance Mode (not connected to any Center Server) 
0.03 days 
SSH is on this host. You should disable SSH unless it is necessary for adrmi 
You are 
using ESXi in evaluation mode. This license will expire in 60

6. Find a location of a .zip package on ESXi (use find / -name “*.zip” command)

ssh root@10.10.10.5 — -ssh -l root 10.10.10.5 — 123x4 
[Password: 
The time and date of this login have been sent to the system logs. 
WARNING: 
All commands run on the ESXi shell are logged and may be included in 
support bundles. Do not provide passwords directly on the command line. 
Most tools can prompt for secrets or accept them from standard input. 
VMware offers supported, powerful system administration tools. 
Please 
see www.vmware.com/go/sysadmintools for details. 
The Shell can be disabled by an administrative user. See the 
vSphere Security documentation for more information. 
[(root@WN-ESXi7:-] 
[ [ root@WN 
root@WN 
[[root@WN 
-ESXi7:-] 
-ESXi7:-] 
-ESXi7:-]

7. Run esxcli software vib install -d /path/to/the offline bundle 

find / —name zip" 
/usr/lib/vmware/esxupdate/systemstorage . zip 
/vmfs/v01umes/Seeeef72-b3de1fØb-16f7-1c697a624d1c/ESXi6SØ-VYKUSB-Nrc-FLING-332681Ø2-off1ine_t 
/vmfs/v01umes/5eeeef72-b3dØ1fØb-16f7-1c697a624d1c/ESXi7eØ-VMKUSB-NIC-FLING-34491Ø22-componen1 
root@WN-ESXi7:- 
esxc 
so tware VI 
nsta 
vm s vo umes Seeee 72 3 el e 16 7-1c697at

8. Reboot your ESXi with a new USB NIC connected (Web Client > Host > Reboot)

vmware ESXü 
Navigator 
Manage 
Monitor 
6] Wtual Machhes 
Storage 
WN-ESXi7.O 
G) Get vCenterServer I b Create/Register VM I 
I Refresh I 
WN-ESXi7.O 
Version: 
Uptime: 
7-0.0 (Build 15843807) 
Maintenance Mode (not connected to any Center Server) 
0.03 days

That’s it!

Your external network adapters should now be natively supported in ESXi 6.5, 6.7 or 7.0 run on 10th Generation Intel NUC.

I have tried two different adapters, Belkin F2CU040btBLK USB-C and some very old USB-A 3.0 Adata one – both worked perfectly. I left Belkin plugged in as it’s much newer and 2 weeks after switching to NUC I can confirm that I’ve not had a single network performance issue. My USB-C wired NIC is happily used as a WAN interface by Untangle Firewall VM. Happy days!

Lastly, if you need Intel NUC 10th Gen ESXi image without having PhD in VMware PowerCLI and Google, give us a shout and we will share the image, hopefully saving you some time!

Tons of love,

WiFi Ninjas x

WN Blog 023 – Indoor RTLS with WiFi and BLE – Deep Dive

Hey! Welcome to our latest WiFi Ninjas Blog 🙂 We’ve been busy lately testing some available RTLS solutions from a few different major wireless players and fell absolutely in love with the topic to the point, where we shake from overexcitement when thinking or talking about it ????

This summarises how we feel:

Excited Ninjas

Steve Jobs once said, “people don’t know what they want until you show it to them”. He was right.

20 years ago, WiFi was “a nice to have”. 18 years ago, paper maps were at peak of their popularity, until GPS receivers got small enough to be put in the handheld devices. Is anyone not using Google Maps on their mobile? 15 years ago, not many people felt they needed a wireless headset, until Bluetooth was introduced. 7 years ago, NFC was a niche. Today, most of us use contactless payments using cards, phones or watches.

Indoor wireless RTLS is at the early adoption stage. Technology is available, we just need to make it more interesting and rewarding to use.

Satellite-based location positioning services are not always practical indoors. We could offer wireless RTLS functionality based on WiFi, BLE or soon, possibly, Ultra Wide Band (UWB). Modern smartphones have both WiFi and BLE radios built in. Additionally, new iPhones support UWB. Wireless vendors leverage cell of origin, trilateration, triangulation, angle of arrival, proximity and more with WiFi, BLE, UWB, GPS, NFC, mobile networks or any combination of those methods and technologies, offering different levels of accuracy and functionality. RTLS market is growing fast – according to Bluetooth.com, 1.7 billion devices will use indoor BLE RTLS by 2023, translating to even 500% increase in BLE RTLS implementations in some verticals!

So, what do we need to see the RTLS market explosion? We’ll be speculating, but our hearts tell us that indoor RTLS needs solve real challenges like indoor wayfinding or call for help and simply be more interesting not only for the business’ IT and marketing, but for everyone. We also need more skilled wireless professionals willing to embrace broader wireless technologies and emerging use-cases, like BLE and RTLS, on top of WiFi design, security and analysis skills, while not forgetting about programmability elements.

But enough of the babbling, let’s cut straight to the juicy content! Here is what we will cover in this blog:

Blog Structure:

  • Theory
    • RTLS Technologies
      • WiFi
      • BLE
    • RTLS Tracking Methods
      • Cell of Origin
      • WiFi Trilateration
      • WiFi Angle of Arrival
      • Mist vBLE Arrays and Probability Surfaces
    • RTLS Functionality
      • Presence & Analytics
      • Location
      • Engagement & Actions
    • RTLS RF Design
      • Design Tips
      • Examples
  • Demo Time!
    • Test Environment
    • WiFi Trilateration
    • WiFi Hyperlocation
    • Mist BLE Arrays
  • Location API
    • Challenge 1 (Meraki Location API): Do You Need RTLS RF Design for WiFi Location & Presence Analytics?
    • Challenge 2 (Cisco CMX Location API): Can You Leverage Enterprise Messaging Solutions to Benefit from Indoor RTLS?
  • Gotchas

Theory

RTLS Technologies

WiFi

  • Most popular tracking technology, as it also provides access to the network.
  • Can offer different levels of location accuracy, depending on the tracking method used (cell of origin, Trilateration, AoA, etc.).
  • Multiple element antenna arrays can substantially improve location accuracy – Cisco Hyperlocation is a great example.
  • Can be used to track associated and unassociated probing clients.
  • Associated is proffered, since stations are chattier (with their screen on at least).
  • Normally, WiFi based RTLS is based on RSSI/Location information from probe requests only, as probing client normally sends requests on multiple channels, and is therefore seen by multiple APs. This, however, results in very infrequent location calculation ranging from 10 seconds to 5 minutes (according to Cisco) but our own tests shown that modern unassociated stations don’t probe at all. Note, that happy (high RSSI/SNR) associated client probably won’t probe at all, and therefore its location could not be calculated with better accuracy than ‘cell of origin’ (see below) until its re-associating (roaming). Cisco has addressed this challenge with a feature called ‘FastLocate’. It uses a dedicated built in antenna array (4800) or a halo module (3600 / 3700) to scan multiple channels and get RSSI values from clients’ data packets across multiple channels without sacrificing performance of clients serving radio by going off-channel to do just that.
  • To track unassociated station, two conditions must be met: station must not use MAC randomisation (they normally do) and it must be probing. Not all unassociated stations are probing.
  • Client apps are optional and can be used to increase tracking accuracy, location calculation frequency and add engage or actions element. Note, that using mobile SDK in WiFi-based RTLS is challenging, as the application normally has no way of knowing the MAC address of the device it sits on.
  • On top of location analytics, WiFi is normally used to provide presence analytics (more info below).

BLE/vBLE

  • BLE relies on physical beacons, often battery operated; vBLE moves beacon role from BLE beacons to APs .
  • BLE is often used to offer proximity-triggered actions and vBLE can be used for location analytics on top or instead of WiFi
  • Client app is required for fast (sub-second with Mist) and accurate (1-3 metres) BLE/vBLE. Station listens to the BLE transmissions coming from the BLE AP (BLE APs are transmitting). Mobiles listens to all (directional with Mist) beams from all BLE APs in the area and sends RSSI values with beams ID to the server/cloud, where location engine will calculate station XY coordinates and send them to the station. Note, that mobile station can send that information using mobile network; device doesn’t have to be associated using WiFi for BLE RTLS to work.
  • Clients with no apps will be treated / located as BLE tags / assets. In this scenario, client is sending BLE transmissions to BLE APs (BLE APs are receiving). Mist uses its directional arrays to pinpoint asset tag transmission location with mind-blowing accuracy, all other vendors will typically rely on tag proximity to BLE radio built into the AP / anchor.
  • More BLE beams heard = better accuracy.
  • More element antenna arrays can substantially improve location accuracy – Mist BLE Arrays are a great example.
  • BLE seems to be very well suited for RTLS, as it doesn’t travel as far as WiFi, therefore offering theoretically better accuracy than WiFi. Note that both BLE (2.4GHz) and WiFi (2.4 or 5GHz) are using the same frequency (actually WiFi can even sit on a higher frequency with 5GHz, and therefore theoretically not travel as far as 2.4GHz) but BLE (Bluetooth LOW Energy) uses significantly lower power levels to operate.
  • XY coordinates are calculated for vBLE and are typically not calculated for BLE (proximity used instead, but there are exceptions to that rule!).

RTLS Tracking Methods

Cell of Origin

  • Simplest location technique.
  • Typically leveraging location of AP that clients are connected to but might be the AP that sees the probing or associated client with strongest RSSI.
  • Great for simple zone-wide accuracy.
  • Requires at least 1 AP per zone and careful RF design.
  • XY coordinates are not calculated.

WiFi Trilateration

  • Distance (lateration) based location technique using RSSI in 802.11, measured by either STA or AP/Sensor.
  • Adds more accuracy within a zone (tested with Meraki and Cisco) – 2-3m labbed (in perfect environment), 5-7m marketed, 7-10m real.
  • Real accuracy is lower than labbed and marketed, as it’s difficult to have LOS between 3x AP and STA everywhere. Also, cross floor leakage, atriums, walls and signal deterioration – all affect accuracy.
  • Note, that we normally use Trilateration in WiFi RTLS and it’s often confused with Triangulation.
  • Trilateration: requires 3 APs with known distance between them; uses RSSI (distance, lateration) to calculate intersection (client XY coordinates) between three circles (as shown above).
  • Triangulation: requires at least 2 APs with known distance between them; uses this baseline and multiple-element antenna arrays (that we typically don’t have in WiFi APs, with Hyperlocation being exception here) to calculate arriving signal angles to find XY coordinates of the client (as shown below).
  • XY coordinates are calculated.

WiFi Angle of Arrival

  • AoA (angulation) location technique using angle of incidence at which STA signals arrive at the receiving sensors.
  • Requires at least 2 APs / sensors / modules (at least 3 recommended for better accuracy).
  • Note, that AoA with three APs is in fact a tri-angulation.
  • Requires multiple element antenna arrays or antenna mechanical agility.
  • Cisco Hyperlocation uses a mix of WiFi Trilateration (RSSI), WiFi AoA and BLE
  • Cisco 3600/3700 Hyperlocation module has 32-element antenna array
  • Cisco 4800 built-in Hyperlocation has 16-element antenna array
  • According to Cisco, Hyperlocation module used with older APs and 4800 built-in array offers same levels of accuracy
  • AoA is more accurate than Trilateration (tested with Cisco Hyperlocation) – 1-3m labbed, 1-3m marketed, 1-5m real
  • Real accuracy is lower than labbed and marketed for similar reasons as discussed for Trilateration. Additionally, AoA requires extremely careful mounting and calibrating APs positions in maps services (height, azimuth).
  • XY coordinates are calculated.

Mist vBLE Array and Probability Surfaces

•aoo 
BEAMS 1 HEAR 
CLOUD
  • Unique to Mist.
  • Every Mist AP has 16 Directional Antennae Bluetooth Array – 8 reflectors and 8 directional antennas.
3-
  • Uses BLE concept, where Mist SDK on the mobile device is listening the beacons from the beams and sends the RSSI and device sensor information back to the Mist cloud through either WiFi or cellular. Mist also supports assets tracking.
  • Mist is not using standard Trilateration, but ‘Probability Surfaces’, where combination of listening to directed AP Beams and machine learning constantly evaluating Path Loss Formula (PLF) per device to calculate station location.
  • Extremely good accuracy: 1-2m labbed, 1-3m marketed, 1-3m real (mind blasting).
  • XY coordinates are calculated.

RTLS Functionality

Presence & Analytics

  • Typically relies on WiFi
  • Provides network level stats, such as:
    • Current visitors: devices count, dwell time, gender and age split, device types present
    • Avg. visit distribution: time of day, day of week, new vs repeat, duration, gender, age
    • Bounce rate (stayed vs bounced; enter and stay for longer than 3 min)
    • Conversion rate (converted vs passed; clients that passed the venue but didn’t enter)
    • Visitors engagement ratio (stayed connected over specified time)
  • We can leverage social media, mobile apps or splashpage forms to onboard WiFi clients and grab users’ details (be careful with GDPR!)

Location

  • Typically relies on WiFi, BLE/vBLE, GPS, Mobile, UWB, Ultrasounds
  • Provides zone/location level insight, such as:
    • Zone analytics & zone paths
    • Location on the map (dashboard or client app – blue dot)
  • Some location features can be leveraged directly through specific web user interfaces (CMX, Purple, Mist or Meraki Dashboards) but it’s generally more powerful and useful to get, manipulate, filter and visualise location data using API – output can be crafted to any business needs. Typically, the following attributes can be grabbed (or subscribed to if you’re using Mist Webhooks) via location API:
    • Client XY coordinates
    • Zone entry and exit events
    • Virtual beacon (Mist), beacon proximity (everyone else)
    • Raw tracked client’s data: RSSI, time, MAC, etc.
  • Zone analytics importance
    • Zone analytics is extremely valuable for businesses, especially retail, to understand impact of their actions (new displays, stands, brands, promotions, etc.) on paths users are taking and stats for each zone (dwell time, count, etc.)
  • CMX GUI – Zone Paths example:
A screenshot of a video game

Description automatically generated
  • Purple GUI Zone – Paths example:

Engagement & Actions

  • Uses Presence, Analytics and Location insight to create personalised user experience
  • Example 1:
    • Returning user that has opted in and has previously bought a pizza (tagged with ‘pizza lover’ tag) entering 1st Floor dining area after 3.15pm will get a 50% off offer delivered via App Push Notification (This would also require some CRM integration)
  • Example 2:
    • When user is in a proximity to a BLE beacon or a vBLE area, open wifininjas.net website in a loyalty app integrated browser
  • Example 3:
    • Zone A (sports cars) will use different Captive Portal than Zone B (donuts)
  • Example 4:
    • User uses app for indoor wayfinding – where am I and how can I find client X reception and how to get there?
  • Example 5:
    • User requires assistance in the shop (looking for a certain size of shoes to try); user can use the mobile app to ‘call for help’ and grab attention of staff; staff knows where to find a client calling for assistance as they also use location-aware mobile app

RTLS RF Design

Design Tips

  • APs should be located around zone perimeters to create a convex hull
  • Each client should be within convex hull of at least 3 APs with solid RSSI (-65dBm is OK)
    • It’s ideal for all tracking methods, even though AoA and Mist BLE require just 2 APs to work; still, more APs give better accuracy
  • Ideally, use dedicated radio or module for RTLS to maximise location scanning performance and minimise performance impact to client-serving radios
  • Ensure LOS is maintained between APs and clients (AP behind ceiling tiles is a no-no)
  • Don’t mount APs too high! Not more than 4.5m is OK
  • Validate secondary and tertiary signal strength in your favourite survey tool
  • Cisco specific:
    • Enter thick walls into Prime and ‘Enable OW (Outer Walls) Location’ in CMX to use it for calculation
    • Always use FastLocate functionality (leverage client data packets and probes to calculate location)

Examples

Note, that number of APs has tripled! Generally speaking, if the design is good for RTLS, it’s also good for any other use case (data, voice, most high-density scenarios, etc.). Sometimes some WiFi radios need to be put into non-client-serving mode as too much WiFi can be damaging to the performance of the network (topic for a different discussion).

Demo Time!

Test Environment

  • All APs were positioned quite nicely, LOS with a client maintained everywhere within a convex hull making it slightly unrealistically great (normally possible in lab environment only).
  • Laser tool was used to measure height of the APs.

WiFi Trilateration

Components

  • Tested with traditional Cisco and Cisco Meraki
  • We have deemed Meraki Dashboard unusable for RTLS (shows only point of association and overlays all clients from all floors on every floor map) so we’ve also used Purple as an RTLS Dashboard overlay with Meraki testing
  • Prime (or DNAC) is needed to create maps, place APs, set their height, azimuth, specify zones, inclusion & exclusion areas, walls, rails (wayfinding paths)
  • Theoretically CMX is not required if Hyperlocation is not in use and we could connect AireOS or C9800 WLC directly to the DNAS. Unfortunately, we couldn’t do it as:
    • Direct connection requires DigiCert CA Root Certificate that we don’t have on the WLC
    • C9800 code version 16.12.2 is required, but 16.12.1 was newest publicly available at time of testing

Accuracy

  • Theoretical expected accuracy of 5-7m
  • Better than expected accuracy when inside the convex hull (2.55m average; 2.69m 90% error distance)
  • Still good accuracy when on the edge of the convex hull (2.57m average; 3.38m 90% error distance)
  • Location Computation Frequency of 11-15 seconds (with screen on, associated and active device) is not enough to offer blue dot experience but it’s still good for basic RTLS functionality like simple, static wayfinding or zone analytics with zones sized accordingly to solution accuracy

Cisco Hyperlocation

Components

  • Prime or DNAC still needed for maps
  • Overall complicated setup, prone to user errors and bugs
  • CMX on-prem required to do all data crunching – it’s quite a lot of data to go through!
  • DNAS can be used to access presence, analytics and location data in the cloud if required
  • Note, that at the time of testing DNAS was lagging behind CMX and was generally slower and less accurate than CMX!
  • DNAS can also be used for many more things, like integration with 3rd party tools or Open Roaming integration (amazing idea that we feel will change the way we use public WiFi – let’s leave it for a future blog ????)

Accuracy

  • Theoretical expected accuracy of 1-3m
  • Totally brain smashing accuracy inside the convex hull (0.73m average; 0.73m 90% error distance) for a non-moving client
  • Good accuracy at the edge of the convex hull (1.78m average; 2.03m 90% error distance) for a non-moving client
  • Note, that accuracy of mobile clients will depend on their speed of movement and since location computation frequency is 2.3-2.4 seconds at best, the distance travelled within that timeframe will be typically added to the calculated location error distance

Demo

Let’s look at the video showing CMX app, CMX Dashboard and Matt walking around my house ????

  • Accuracy is generally really good!
  • Blue dot location update (location calculation frequency) of 2-3 seconds is not bad and should be enough for most indoor RTLS use cases.
  • We didn’t draw rails (wayfinding paths) in Prime so what we’ve seen here is raw XY calculations displayed on the map. Rails around the path Matt’s taken would snap him into it, making blue dot appear more accurate.
  • CMX has great zone path visualisations built in and it’s generally a rarity (Purple also has it built in and it’s as good; note that not everyone will leverage GUI-based zone paths analytics since we can use API and visualise it ourselves and integrate it with existing tools but it requires clearly defined requirements and solid skills and effort).

Mist: vBLE Array

Components

  • Yup, that’s it ???? Class leading simplicity. We’ve had a lab running by lunchtime and never had a chance to play with Mist before!
  • All maps management, wayfinding paths, beacons and zones done easily in the Dash.

Accuracy

  • Less than 1m inside the convex hull.
  • Reliable sub second Location Computation Frequency when used with an app, making it difficult for other major WiFi vendors to compete with.

Demo

The video screams million words! Let’s take a look ????

  • Absolutely mind-bending accuracy with quickest location computation frequency we’ve seen to date in indoor RTLS space!
  • It is expected to still use WiFi to get network-wide presence and analytics stats, despite using BLE for location.
  • Mobile app is required for the proper BLE solution to work (might not be a bad thing, since mobile app is required anyway to get the most of any indoor RTLS solution, despite the technology used).
  • Mist has no zone path visualisations built in (we only have basic zone analytics) – it’s expected to leverage API to access / visualise this data.

Accuracy Summary

Below is a quick summary showing results of our tests for different indoor RTLS solutions!

Location API integration with existing infrastructure – two examples!

Meraki API – Cell of Origin Example

Background

We have recently worked on project for a beautiful, listed retail store in London, where following a very successful design and implementation of Meraki data WiFi network in the store, retail RTLS topic has emerged. With RF design crafted for data, we have faced a very real challenge – would the existing placement, type and number of APs be useful for indoor Real Time Location Services with a zone-level accuracy?What would be the challenges, limitations and reasonable expectations in terms of RTLS usage and accuracy? Could Meraki API help achieve goals set by the business?

Challenges

RTLS was not in the original scope and, combined with a store physical environment, we have faced some challenges:

  • RF Design: the network was designed with data in mind. We didn’t have APs placed on the outside coverage zones’ perimeters, had just 1 or 2 APs per zone in most areas and proved with a detailed WiFi survey/assessment that zone accuracy leveraging WiFi trilateration should not be expected.
  • Physical Environment: the building was listed, APs/antennas mounting options limited, walls and ceiling mostly wooden, with very low attenuation. All three sections (east, west, central) had big atriums contributing to cross-floor RF leakage strongly. Building had 6 levels.
  • Vendor Choice: we had Cisco Meraki and Purple already in place. Purple leverages Meraki API and uses Meraki pre-calculated XY coordinates (this can’t be changed) to display users’ location on the map. With data design, we didn’t have enough APs per zone to rely on the Meraki XY coordinates. Trilateration requires min. 3 APs in a zone to work reliably. Normally, we would have just one best AP (strongest RSSI) in every zone, with additional two APs needed for trilateration being located in adjacent zones or even a floor or two away! This has contributed to wildly inaccurate location readings in Purple (approx. 22m 90% error distance, often on a wrong floor) and could not be relied on. Note, that network-wide WiFi-based Presence & Analytics stats were perfectly reliable.

Floor diagrams below show the difference in RF Design for Data (what we have) vs RF Design for RTLS (what we would need for WiFi Trilateration to work):

Expectations

Based on the additional very detailed survey (more than 100 test locations across multiple floors with 3 different device types: flagship Windows 10 laptop, Android phone and iPhone), where we temporarily installed several new APs on tripods, we concluded that achieving satisfactory and reliable zone-location accuracy  (10m 90% error distance) using WiFi trilateration would require to approximately double the number of APs. The business has decided not to install additional APs and knowing that a mixture of Meraki and Purple could not offer reliable location insights and that WiFi trilateration would not be practical to use at all (because of the design, regardless of a vendor), we looked for alternative solutions and new success criteria were defined:

  • Location analytics with zone accuracy was the new goal; access to zone-based stats & clients’ paths deemed critical.
  • Zone sizes were increased; sometimes a zone would be as big as a 30x20m open space, sometimes as small as 8x8m room.
  • Still use Purple for presence & analytics, guest splash page & social media integration.

Solution

We have gathered some facts: data & voice performance over WiFi, capacity, coverage and roaming were all rock solid across all WLANs (MAB/CWA for guests and EAP-TLS machine auth for corp) and for all test devices (company issued laptops, major OSes, newer and older phones). High density areas, like ground floor entrance hall, could get very busy during holiday periods (200+ associated devices in a small 15x20m area) and were covered by three high-gain dual-band sector antennas. 5GHz was almost free of channel contention with careful APs placement around atriums and use of properly tweaked RF Profiles: 20MHz channels width, limited max and min Tx power, disabled data rates of 11Mbps and lower. 2.4GHz was tweaked even more, with several APs having their radios off and max Tx power set to the level, where RSSI was generally lower on 2.4GHz than on 5GHz throughout the store and only OFDM rates were allowed. 2.4GHz was still considered best effort. Presence & analytics with Purple was spotless.

Switching focus to location zone analytics, we have confirmed with a detailed post-deployment survey, that our test clients were reliably associating with APs installed in the zones they were in. WiFi scans from unassociated devices taken in multiple spots in each zone revealed that we could use strongest RSSI reading from one best AP to accurately pinpoint the probing or associated device to the zone it was in. We concluded that we could potentially leverage two attributes to correlate WiFi devices with their current zones:

  • Associated devices only: use MAC address of the AP the device is associated with.
  • Associated OR unassociated probing devices: use MAC address of the AP that reports probing or associated device with strongest RSSI.

Two things to note:

  • WiFi MAC randomisation makes it impossible to track unassociated devices that use it.
  • New mobile platforms battery saving modes make those devices very quiet – they often won’t probe at all.

To further simplify our zone analytics calculations, we have decided to use just one attribute moving forward – MAC address of the AP that reports the probing or associated devices with strongest RSSI.

We have discovered that it is possible to get the RSSI attribute for every client seen by all near APs with Meraki API location data. Readings are provided every minute and contain RSSI values covering last 60 seconds. While it’s not very fast and could not be used for a blue dot experience, it is enough for our use case – historical view of zone paths and analytics.

Using strongest RSSI proved to be 95% accurate across entire building and 100+ test spots when correlating user (reporting AP) location with a zone.

Raw Data from Meraki API – note ap_mac, seen_time, client_mac and RSSI

As we know exactly which zone all the APs are installed on, we could easily generate reports showing clients paths with zone accuracy, time spent in each zone, number of clients per zone, etc. without relying on wrongly pre-calculated Meraki XY coordinates.

To automate the zone paths visualisation and zone analytics, client’s software development team has created a tool to do just that. At this point, imagination was the limit.

Conclusion

Sometimes we are limited by a solution functionality when trying to meet client’s requirements. In our example, Purple could only use Meraki pre-calculated XY coordinates and it was not practical or accurate enough (even for a zone-wide accuracy) to use with a data / voice RF design. Let’s answer our initial question. Do You Need RTLS RF Design for WiFi Location, Presence & Analytics? Short answer is “no” for presence & analytics and “it depends” for location. Presence & analytics provide us with network-wide stats, without the location awareness and therefore will not require RTLS design. Location, however, requires very careful RF design to provide different levels of accuracy. With a solid data RF design, where association and roaming trends are reliably predictable, we could expect solid zone-wide accuracy. Anything more accurate (trilateration, Hyperlocation, vBLE) would require more APs, detailed survey, RTLS RF design, careful placement of APs/antennas and pedantic maps services configuration with spotless APs positions, height and azimuth set.

Cisco CMX API – Location Analytics integration with WebEx Teams Chatbot Example

Background

At Natilik, we’re currently in the process of upgrading our showcase with Cisco DNA. The plan is to have full SDA Fabric, C9800-based Fabric Wireless, proper RF design with Cisco 4800 APs, CMX and DNAS in one part of the office and a mixture of AP43 with BT11 in the other part. We’d love to not only showcase Cisco Hyperlocation and Mist BLE Arrays in action, but also leverage the tech to offer indoor wayfinding with turn by turn navigation and onboard guests using Hotspot 2.0 (called Open Roaming by Cisco) for our visitors WiFi.

As of today, we don’t have the new showcase fully running yet, so all we have to play with is ‘standard’ corporate WiFi design for Voice and Data.

The above shows one out of the two floors we occupy. With current design, all we can use is Cell of Origin and, around reception, WiFi Trilateration. And that’s it. Location Calculation Frequency of 11-15 seconds is not enough to offer blue dot experience, but we thought it could still be good enough to solve some wayfinding challenges!

Challenge

We have Cisco AireOS Wireless with CMX, Cisco WebEx Teams and everyone has a corporate phone enrolled with Meraki MDM.

We would love to use the kit we have today to locate a colleague, zone or a meeting room by asking WebEx Teams chatbot about the location of person/location we’re after without the need to install the mobile application, as we can’t have a turn-by-turn indoor navigation just yet anyway.

Let’s imagine that Mac wants to find Matt (yeah, we don’t hold our hands all the time and sometimes attend different meetings on different floors, lol). How do I do it? How would WebEx Teams chatbot know about my or Matt’s location? How could it show or tell me how to find him? Is it really that helpful with approximately 10-12m 90% error distance accuracy? Can we still draw an indoor map showing where should Mac go to find Matt?

Solution

First, we had to set realistic expectations. We shouldn’t expect the solution to pinpoint user to his or her desk knowing, that calculated user location has 10-12m accuracy. Instead, we have decided to use zones that are big enough to account for that location accuracy.

Here are the zones we’ve created, ensuring that each zone has at least one AP. We’ve also considered expected AP association ensuring it all makes sense:

Now, we need to figure out what happens when Mac is asking WebEx Teams chatbot about Matt’s location.

Wait, what is a chatbot? Good question! Most modern enterprise messaging solutions allow to create custom chatbots with custom functionality. We are fortunate enough to have a proper coding nerd in our ranks, Darren, that finds coding chatbots as easy as you probably find putting your socks on. Darren has created NatBot (I wonder where the name comes from!), that can translate natural language with certain key words to specific actions.

To make it all fly, we’ll leverage Cisco CMX Location API, Cisco Meraki MDM API and a cloud map service (i.e. Mapwize) to visualise the results.

Finally, let’s look at detailed steps that are happening in the background when Mac is looking for Matt ????

  • Mac Deryng asks NatBot via WebEx Teams: “Where is Matt Starling”?
  • NatBot notes who is asking and who is he/she looking for
    • Mac Deryng is asking
    • He’s looking for Matt Starling
  • Find correlation between names and corporate iPhones MAC addresses
    • NatBot leverages Meraki MDM API to find MAC address of Mac’s phone (MDM returns MAC address)
    • NatBot leverages Meraki MDM API to find MAC address of Matt’s phone (MDM returns MAC address)
  • Find location of MAC address on the map and correlate it with a zone overlaid on the map
    • NatBot leverages Cisco CMX Location API to find location / zone that Mac is in (CMX returns zone name)
    • NatBot leverages Cisco CMX Location API to find location / zone that Matt is in (CMX returns zone name)
  • Show results for Mac
    • NatBot displays zone name that Matt is in and displays a map of the floor, highlighting that zone
    • Additionally, NatBot displays a ‘map’ button
  • Mac knows which zone Matt is in, but he’s not sure how to get there; Mac clicks on the ‘map’ button in his WebEx Teams
    • NatBot builds a URL using specific Mapwaze syntax that contains start zone (Mac) and target zone (Matt) and opens that URL
    • Browser displays Mapwaze service, that uses pre-created zones, lifts, staircases and paths and shows exactly how to get from start zone to target zone

Sounds complicated? Watching this short video should clear things up!

Sky is the limit

This is just the tip of the iceberg and we have some more ideas about what to do next, once the new showcase is in! ???? Here is the list:

  • Add integration with voice assistants – why type when can just ask a question?
    • Mac: “Alexa, where is Matt Starling?”
    • Alexa: “He’s in the toilet on the 1st floor. Take a lift or stairs, go right, and right again after 10 metres. You can also see the map on your screen”
  • Add integration with calendar
    • Mac: “Alexa, what’s my next meeting?”
    • Alexa: “Your next meeting is recording podcast with Matt. It’s in Fiennes meeting room on the 3rd floor. You’re also on the third floor. Walk out of the back office and take first left and look for second door on the right. You can also see the map on your screen”
  • Add full wayfinding functionality leveraging mobile SDK
    • Support both Cisco Hyperlocation and Mist BLE
    • Offer full turn-by-turn indoor wayfinding
  • Leverage mobile sensors to enhance the blue dot experience
    • Use phone accelerometer and compass to make the experience smoother
  • Offload indoor location with Hyperlocation to GPS and 5G
    • Why limit yourself to just inside?

Gotchas 

Lastly, here is some stuff that we think might save you some ball ache ????

  • Location Computation Frequency for your mobile device will be affected when your screen is off or when your phone goes to sleep
    • Screen off (WiFi-based RTLS): typically doubles the time needed to calculate device location; normally your mobile will go to sleep after several seconds with screen off unless some apps running in the background require connectivity
    • Sleep (all methods): typically your device can’t be tracked when asleep; if apps are used, they would have to be excluded from battery saving modes for tracking to happen
  • NTP
    • Always use NTP – AoA won’t work when AP, WLC or CMX have even slight time differences
  • Components Compatibility
    • Always check compatibility matrix!
    • Newest software across the board (CMX, Prime, WLC) does not mean it will work (we’ve learned it the hard way)
  • APs Mounting
    • Mount APs carefully, especially if multi-element arrays are in use; make sure they’re level!
  • Maps Services Fine-Tuning
    • Set APs exact locations, height and azimuth in maps services! It MUST be spotless for the solution to work
  • Don’t mix Hyperlocation with non-Hyperlocation APs
    • Device associated with a non-Hyperlocation AP on will always be shown as ‘RSSI’ (and not ‘AoA’) in CMX, even if there are Hyperlocation APs nearby
  • Associate WiFi Clients
    • Modern devices won’t probe when not associated (with WiFi on), so tracking unassociated devices, in most cases, provide very little value (and even less with WiFi MAC randomisation)
  • Use Mobile Apps
    • Optional for WiFi, but can increase accuracy and sampling frequency; mandatory for BLE
  • Add C9800 to CMX as ‘Unified WLC’ using SSH, as opposed to ‘WLC’ using SNMP 

You’ve made it! This blog has almost 6000 words. Well done! ????

With tons of love,

WiFi Ninjas x

WN Blog 022 – Distance Between 802.11 Radios – How Close Is Too Close?

We have recently come across few production networks, where distance between two APs or APs and a client stations was much closer than I was comfortable with.

Natural reaction was to suggest moving radios at least few metres apart. But why exactly? What happens when two or more transmitting radios are too close to one another? How would placing a laptop next to the AP affect wireless network quality?

Intrigued and eager to answer questions that, interestingly, no one was asking, I have decided to lab it up, research it more deeply and document my findings. I quickly realised that there is not one, but two issues potentially affecting wireless transmission, where distances between radios were too short. Let’s discuss Channel Leakage and Near-Field Interference.

Structure:

  • Channel Leakage
  • Near-Field Interference
  • Literature

Channel Leakage

Few months after a successful design and refresh of a WiFi estate for a financial institution in London, I came back to work on a periodic wireless survey and to assess if there is anything that would be potentially stopping them from introducing more agile working heavily relying on the new WiFi deployment.

I was told that there is a separate company that is working closely with my client in the same offices and that they use their own, separate WiFi network. They have decided to put their own APs next (few inches away) to our APs convinced that since it worked perfectly for my client, it would continue working when two overlaid networks operate simultaneously. This is how the new original vs new deployment looked like:

Having completed the survey, we have concluded that city centre location inside a multi-tenant building with multiple WiFi networks leaking from the outside and adjacent floors combined with two separate, overlaid WiFi networks contributed to very high CCI averaging at 10 or sometimes more. RF tuning across both networks has helped a lot with the contention but the overall quality of the WiFi was still not great. It felt slow despite having strong underlying infrastructure, little CCI and fast Internet pipe. Quick wireless capture has revealed excessive retransmissions rates peaking at 30% in some areas even when there were not too many clients (maybe 10 per radio, often less) competing for the airtime, channel utilisation peaking at 10% and with no obvious faults with the configuration.

Lab

Next step was to reproduce the issue in the lab, where I wanted to show how channel separation and AP-to-AP distance impacts percentage of retries in the wireless transmission.

Setup

  • 2x Cisco 3702i APs registered to C9800-CL WLC broadcasting separate BSSIDs on 5GHz only
  • 20MHz static non-overlapping, clean channels (100 & 104 and 100 & 140) and max Tx power (1) set
  • One mobile station associated to each BSSID, running external speed test in the loop
  • Ekahau Sidekick used for wireless captures

Tests

4 different tests were performed, each test involved wireless capture over 30 seconds and was repeated 5 times to minimise measurement errors:

  • Channels 100 & 104 (no channel spacing)
    • Test 1: APs 0 metres away from one another
    • Test 2: APs 3 metres away from one another
  • Channels 100 & 140 (200MHz channel spacing)
    • Test 3: APs 0 metres away from one another
    • Test 4: APs 3 metres away from one another

Conclusion

We can clearly see in Test 1, that close physical distance between radios combined with no channel separation resulted in 15.4% retries.

Introducing 200MHz channel separation (Test 3) without extending distance between radios has reduced retries ratio by about half, to 8.2%.

Moving APs 3 metres away from one another has further reduced the retries to 3.9% (Test2; no channel separation) and 2.3% (Test 4; 200MHz channel separation).

Unplanned channel leakage really is an adjacent-channel interference, even though our channels theoretically don’t overlap. ACI will naturally cause increased number of retries, and therefore decrease the throughput and contribute to the slow WiFi perception.

Note: we used flagship Cisco APs with great quality of components providing great RF accuracy. Using cheaper APs that pack cheaper antennas and radios would result in less stellar performance of spectral masks (algorithms applied to the levels of radio transmissions used to reduce main channel leaking to adjacent channels) and amplify the effects of intermodulation (signal modulation on two or more different, non-harmonic, frequencies), increasing effects of ACI and percentage of retries when reasonable channel separation and physical distance between AP radios are not maintained.

Near-Field Interference

Another distance issue that I have frequently seen in an enterprise environment is placing receivers too close to transmitters.

Let’s start with a quick definition of what near-field is in wireless.

Near-field is a 1 wavelength region, where electromagnetic field EM charges and electric charge effects are extensively produced, potentially negatively affecting quality of the received transmission within this region.

Near-field interference decreases drastically, in a logarithmic fashion, when the receiver is moved farther away from the transmitter. It is normally considered enough to be 1 wavelength away from the transmitter to negate the impact of near-field interference. Near-field also affects reflected signal for Rx antennas.

Based on the above, we can conclude that:

  • Radios should never be positioned closer than 1 wavelength apart from one another
  • Receive antennas should not be positioned closer than one wavelength to any reflecting obstacle due to reflection indicted multi-path phase cancellation

How far is 1 wavelength? It depends on the frequency. The higher the frequency the shorter the wavelength. Here are the wavelength calculations for our beloved 802.11 bands:

2.4GHz frequency wavelength = 12.5cm

5GHz frequency wavelength = 6cm

Now we know, that when we position 2.4GHz radios closer than 12.5cm away from one another (or 6cm in 5GHz) we would suffer from extensive near-field interference and that the situation would improve greatly when we start increasing this distance. Bear in mind, that unwanted channel leakage might contribute to packets corruption here too, so it typically is recommended to keep at least few metres distance between the radios!

Couple of examples, where near-field interference can affect quality of WiFi transmission:

  • Enterprise-class AP placed on a small desk with few people sitting around it and their laptops virtually adjacent to the AP; note, that external antennas can increase the negative effect of a near-field interference
  • Capturing wireless packets with a capturing device positioned to close to a client or an AP

We must remember that WiFi is not NFC! ????

Lab

In this test I targeted impact of near-field interference on wireless receive quality, where receiver (AP in a sniffing mode) was within 1 wavelength of a transmitter (AP serving clients).

Setup

  • 2x Cisco 3702i APs registered to Cisco 3504 WLC and one test client associated to client serving AP
  • AP with blue LED is serving clients
  • AP with green LED is running as a sniffer and capturing packets on the same channel as the AP serving clients
  • Below tests are based on 5GHz, but in my tests I could have easily reproduced them across both 5GHz and 2.4GHz bands

Tests

It was enough to perform a very simple test here – check captured packets integrity in two scenarios:

  • Test 1: AP sniffer placed 60 centimetres away from a client-serving AP (more than 1 wavelength distance between radios)
  • Test 2: AP sniffer placed on top of a client-serving AP (less than 1 wavelength distance between radios)

Here are the results:

Test 1 captures, where APs are fairly far away from one another, are clean. Putting transmitting and receiving radios very close together (Test 2) results in corruption of almost all packets transmitted by client-serving AP.

Conclusion

When receiver is placed too close to transmitter (especially true when the distance is less than 1 wavelength apart), near field interference will cause packets to lose integrity and result in failed Frames Check Sequence (FCS).

Summary

As shown in the above tests, unwanted channel leakage can seriously affect the transmission quality over distances less than few metres, especially without proper channel separation on the neighbouring BSSIDs. Near-field interference can cause corruption of most of the transmitted frames, where distance between transmitting and receiving radios is less than 1 wavelength apart. Finally, after having a chat with my friend Nigel (Twitter @WiFiNigel), we have concluded that the Rx overdrive might also play a role in the frames corruption. Unfortunately, it is difficult to ascertain which phenomena impacts the frames corruption most using the home lab, so I’ll just conclude with this: don’t stack APs! 🙂 And put them away from strong reflectors. All above issues can be easily avoided by using good quality enterprise equipment, solid RF design and having a high-level understanding of how the distance induced interference can affect the quality of the wireless transmission.

Literature

WN Blog 020 – Postman Variables

In our previous blog (please start there), we were typing in all the site IDs, WLAN IDs, API authentication tokens & Mist API HTTPS URL manually. It’s no fun, right?

Since we’re likely to interact with Mist API using one Mist account, we’ll probably use one API key globally.

Additionally, we’ll probably interact with specific organisations, sites or attributes (like WLANs) that are more ‘local’. It might be separate clients in a managed service environment or separate sites under one org. Whatever it is, it makes sense to think of it as a specific, local environment, something that is not global.

Surprise! We can define our variables on both “Environment” and “Global” level in Postman.

API token will be set as a “Global” variable, and all the rest attributes we used so far will be defined inside an “Environment” called “Mist WiFi Ninjas”.

Mission:

  • Use Global and Environment Variables to make our lives easier

Environment Variables

First, note that we are not using any environments right now.

Our request still has long, manually pasted streams representing site, WLAN and Mist API URL.

Postman 
File Edit View 
Import 
Runner 
Q Filter 
History 
% MyWorkspace • 
GET httpswapi.mist.corn"api/...• 
Invite 
GET httpswapi.mist.com//api.. 
Collections 
GET httpswapi.mist.conwapi/.... 
APIs 
Untitled Request 
Clear all 
Save Responses 
Today 
https://api mist.com.'/api/vl 'sites,'0fc98886-fb c48ef-b905-819200d57d54/wIans,'a88d6cdd-6aa5-4eb20... 
Upgrade 
Save 
Headers (9) 
Test Results 
Visualize 
Body 
pre•request Script 
VALUE 
Value 
Tests 
Settings 
DESCRIPTION 
Description 
status: 2000K Time: 1 71 ms 
Cookies 
Code 
GET 
GET 
PUT 
https://api.mist.com//api/vl 'sites/Ofc 
Params 
98886-fb 1 c-48ef-b905-819200d57d5 
4/wlans/a88d6cdd-6aa54eb2-8cd9-fb 
Authorization 
Query Params 
https://api.mist.com//api/vl 'sites/Ofc 
98886-fb 1 c-48ef-b905-819200d57d5 
4,'wlans/a88d6cdd-6aa5-4eb2-8cd9-fb 
Key 
Body Cookies Headers (16) 
Size: 451 KB 
Bulk Edit 
Save Response 
{(wIan_5ghz)} 
((wlan_5ghz)) 
{(m nsJ 
((wlan_5ghz)) 
https:,'/api mist.com//api/vl 'sites/Ofc 
98886-fb 1 c48ef-b905-819200d57d5 
https//api mist.com//api/vl 'sites/Ofc 
98886-fb 1 c48ef-b905-819200d57d5 
https„'/api mist.com//api/vl "'sites/Ofc 
98886-fb 1 c-48ef-b905-819200d57d5 
Pretty Raw 
Preview 
"class": "best effort" 
"overwrite": false 
id": "d2b94bf9-22bø-4af1-aød1-68e162fbe199" 
"dtim": 2, 
"hide ssid": false, 
"acct servers": 
"created time": 
"disable_uapsd" : 
[1, 
1557492085, 
false, 
"allow mdns": false,

Let’s create our new Environment and name it “Mist WiFi Ninjas”

GET https:,',/agi.rr 
Untitled Request 
Headers (8) 
MANAGE ENVIRONMENTS 
GET https:'%vj•_mi 
)/vvlans/K• 
GET 
An environment is a set of variables that allow you to switch the context of your requests Environments can be shared 
Mist WiFi Ninjas 
Send 
ok 
Save Response 
bet%'een multiple workspaces. Learn more about environments 
Mist WiFi Ninjas 
Globals 
Share 
Import

Define our variables on the Environment level

Postman 
File View 
History 
Today 
Help 
GET 
Untitled Request 
GET 
% My Workspace 
GET https:napinis con-Ina; 
Invite 
GET https:.'/api_mist.com//api„, 
Mist WiFi Ninjas 
https://api.mist.com//api/vl/sitesJOfc98886fb 1 c 4e 52 •8, , , 
Send 
https:.'/api.mist.com//api/vl'sites/Of 
Params 
9200d57di 
GET 
"wIans/aggd6cdd-6aaS-4eb2-8cd( 
I 'sites," 
98886-fbl 9200d5 
4/wIans/aggd6cdd-6aaS-4eb2-gc 
((y.'Ien_Sghzp 
gg8g6-fh 1 c-4gef-hg05-g1 9200dS 
4/wIans/e88d6cdd-6ae54eb2-8c 
https://api.rmst.com//api/vl'site 
MANAGE ENVIRONMENTS 
Environment Name 
Mist WiFi Ninjas 
VARIABLE 
'ave Response 
INITIAL VALUE O 
a88d6cdd-6ass-4eb2-... 
0 08886- 
https://api.mist.com//... 
CURRENT VALUE O 
Persist All 
Reset All 
wlan 
site 
uc ton 
a88d6cdd-6ea5-4eb2-8cdg-fb8731ab18e2 
0 08886- lc-48e- 905-819200 57 54 
https://api.mist.com//api/vl 
Add a new variable

Global Variables

Go to Globals

File Edit View, 
History 
Today 
• My Workspace • 
GET https:\' 
Untitled Request 
Invite 
GET "mist}}. 
Mist WiFi Ninjas 
https://ap...X 
Clear all 
GET 
1 1 9200d57d54,'wlans/ s-4eb2-E 
Send 
PUT 
PUT 
PUT 
PUT 
PUT 
PUT 
vrwIan_5ghzp 
jJwIan_SEh7P 
https://api.mist.comnapi/vl/site 
a88d6cdd-6aa5-4eb2-8c 
https://epi.mist.cum,qapi/vl/site 
98885fb1c48eft905819200d5 
4,'wIans/a88d6cdd-6aa5-4eb2-8c 
' 'nlan_5ghz} 
unlan_5ghzp 
UwIan_5ghzp, 
https://api.mist.com//api/vl/site 
https://api.mist.com//api'vl/sire 
• 'wIans,'a88d6cdd-6aa54eb2-8c 
https://api.mist.comnapi/vl/site 
https://api.mist.com//api/vl/site 
ggg86-fh1c-4gef-hgos-g1g200ds 
• 'wIans/a88d6cdd-6ea5-4eb2-8c 
https://api.mist.comnapi/vl/site 
qggg6-fh1 q200dS 
https://api.mist.com//api/vl/site 
9ggg6-fb1 c-48ef-b90S-819200dS 
https://api.mist.com//api/vl/site 
woods 
d/"' lans/a88d6cdd-6aa5-4eb2-8c 
https://api.mist.com//api/vl/site 
https://api.mist.comnapi/vl/site 
d/wIans/a88d6cdd-6aa5-4eb2-8c 
https://api.mist.com//api/vl/site 
• 'wIans/a88d6cdd-6aa5-4eb2-8c 
https://api.mist.comnapi/vl/site 
98886 f'01c48ef4-,905819200d5 
MwIans/a88d6cdd-6aa5-4eb2-8c 
https://api.mist.comnapi/vl/site 
• iwIans/a88d6cdd-6aa5-4eb2-8c 
https://epi.mist.com,qapi/vl/site 
98885fb1c48eft905819200d5 
MANAGE ENVIRONMENTS 
An environment is a set of variables that allow you to switch the context of your requests. Environments can be shared 
between multiple Learn more about environments 
Mist WiFi Ninjas 
Globals 
Share 
Import 
Yesterday

Define your global token.

Remember to include “Token ” prefix before pasting actual token value.

MANAGE ENVIRONMENTS 
Global variables for a workspace are a set of variables that are always available with 
can be viewed and edited by anyone in that workspace, Learn more about globals 
Globals 
VARIABLE 
to en 
Add a new variable 
INITIAL VALUE O 
CURRENT VALUE O

Replace our manual values with variables

%stman 
File Edit View 
Q Filter 
History 
Import 
Collections 
Runner 
AP IS BETA 
Clear all 
GET https:/,'api.... 
Untitled Request 
GET 
httpswapi.... 
My Workspace • Invite 
GET httpswap... 
GET {Cmistmsi.... 
Mist WiFi Ninjas 
GET {{mist»/si... 
Save Responses 
Today 
Pa rams 
0 
Code 
Headers (9) 
Test Results 
Visualize 
Body 
Cookies 
Presets 
GET 
GET 
GET 
PUT 
PUT 
PUT 
GET 
GET 
{Cm ({site_ ns} 
{(wlan_5ghz)) 
{(wIan_5ghz}) 
{(wlan_5ghz)) 
https://api.mist.com//api/vl/sites/ofc 
98886.fb1 c48ef.b905-819200dS7dS 
4/wIans/a88d6cdd-6aa 54eb2-8cdg-fb 
httpsWapi mist.com/,'api/vl'sites/Ofc 
98886-fbl c48ef-b905-819200dS7dS 
4/wIans/a88d6cdd-6aa 54eb2-8cdg-fb 
{Km 
{(wlan_5ghz)) 
{Cm nsJ 
{(wlan_5ghz)) 
{(wlan_5ghz)) 
https:/fapi 
98886-fb1c48ef-b905-819200d57d5 
https:/fapi mist.com/,'api/vl'sites/Ofc 
98886-fb1c48ef-b905-819200d57d5 
https://api 
98886-fbl c48ef-b905-819200d57d5 
https://api.mist.com//api/vl'sites/Ofc 
98886-fbl c48ef-b905-819200d57d5 
https://api.mist.com//api/vl'sites/Ofc 
98886-fbl c48ef-b905-819200d57d5 
https://api.mist.com//api/vl"sites/Ofc 
98886-fbl c48ef-b905-819200d57d5 
4/wIans 
Authorization 
Headers (2) 
Authorization 
Content-Type 
Key 
Temporary Headers (7) O 
Body Cookies Headers (16) 
Pre-request Script 
VALUE 
"token}) 
application/json 
Value 
DESCRIPTION 
Description 
Time: 687ms 
Bulk Edit 
Status: 2000K 
Size: 452 KB 
Save Response 
Pretty 
Raw 
Preview 
"class": 
"best effort" 
"overwrite": false 
"org_id • 
"d2b94bf9-22bØ-4af1-aØd1-68e162fbe199", 
"dtim": 2, 
"hide ssid": false, 
"acct servers": 
"created time": 1557492085, 
"disable_uapsd": false, 
"allow mdns": false, 
"apply_to": "site", 
"app_limit": { 
"apps" • 
"enabled": false, 
"wxtag_ids . 
. "a88d6cdd-6aa5-4eb2-8cd9-fb8731ab18e2" , 
"vian id": 
"wxtag_ids " : 
null, 
"mxtunnel id": null, 
"ssid": 
"vlan_pooling": false, 
"wlan limit down": 20ØØØ, 
"wxtunnel id": null,

Enjoy Postman environment that is easier on the eyes! Easy peasy lemon squeezy!

With love,

WiFi Ninjas x

WN Blog 019 – Getting Started with Mist API

Let’s face the truth. We can’t avoid dev stuff in WiFi forever! REST API, Python, JSON, Ansible and whatever else is needed – here we come!

It’s time. This is day 1 of our studies!

The mission for today? Actually we have a few 🙂

  • Prepare the environment to work with Mist REST API using Postman
  • List all WLANs created in WiFi Ninjas site using Mist API using GET
  • Narrow the list down to a specific WLAN – “WiFiNinjas(.)(.)”
  • Rename “WiFiNinjas(.)(.)” WLAN to “WiFiNinjasTitties” using PUT

Extreeeemely complicated, we know 😉

Let’s assume everyone knows what the API is. In case it’s not the case, here is a quick read:

https://www.howtogeek.com/343877/what-is-an-api/

Mist (and most other Enterprise solutions) use REST:

https://restfulapi.net/

Lastly, from Mist Documentation:

https://www.mist.com/documentation/category/api/

Steps to complete our little task below!

Install Postman

You can get it here:

https://www.getpostman.com/downloads/

Generate Token

Token is needed so the Mist Dash can authorize us to access API.

Log into Mist Dash, then go to and hit ‘POST’:

https://api.mist.com/api/v1/self/apitokens

All 
api.mist.com/api/vl/self/apitokens 
Lab Natilik WN 
CWNP Gml 
GM T LN Is 
Django REST framework 
Self ! Getall Create Apitoken 
Getall Create Apitoken 
POST /api/vl/self/apitokens 
HTTP 2øØ 0K 
Allow: POST, OPTIONS, GET 
Content—Type: i cation/ j son 
Vary: Accept 
"created time": 1574783690, 
"last used • • : 
null, 
" pc cml , 
OPTIONS 
GET 
Media type: 
Content: 
application/json

If you hit ‘POST’ again, new token will be generated!

Take note of the “key” – it’s our token.

Let’s assume we want to list created WLANs in the test ‘WiFi Ninjas’ site.

Check API URL to list Site WLANs

Mist API is nicely documented. You can go to Mist API documentation page to check the URL for everything that’s possible using RESTful API:

https://api.mist.com/api/v1/docs/Home

api.mist.com/api/v1/docs/Home 
Natilik 
CWNP Gml 
GM 
TOC 
Overview 
Overview 
permiesion 
Query 
Rate Limit 
Mist GithUb Repo 
Authentication 
Login 
Privileges 
Register 
AudiU.ogs 
Integration 
Site 
WIan 
Map
C api.mist.corn/api/vl /docs/Site#wlan 
Lab Natilik WN 
CWNP Gml 
WIan 
Wireless LAN definition 
WIans are the wireless networks created under a site. 
Base URI 
Example 
"ssid"• "corporate" 
"auth": { 
"type": "psk 
// type 
is psk 
"enable 
mac auth": false, 
GM

We now know the syntax to get our WLANs.

Get Org ID and Site ID

Next we need to know the “WiFi Ninjas” Site ID to check against.

Go to Mist Dash. You can get Org ID and Site ID from the browser URL:

All Lab Natilik WN 
Inbox - mac.deryng@gmaiLcorn X 
Mist *stems Administration 
Access Points 
• WiFi Ninjas 
Mist University I Al for IT Mist AP 
SITE ID 
IP Address 
x 
C •a0d1 
CWNP Gml @ G A GM T LN 
ORG ID 
site Buckton Fields 
MAC Address 
pp 
MiSt 
Monitor 
Marvisw 
O) Clients 
Access Points 
Switches 
Location 
Analytics 
CJ Network 
organization 
WIFI NINJAS 
o 
Filter 
Site I API I Mist 
NO. Clients 
X Getall Create Apito 
uptime 
There are no APS found in this site 
Claim your Access Points by providing claim codes 
Claim APS

We now have everything we need to list our WLANs in ‘WiFi Ninjas’ site!

Use obtained API token, WLAN GET URL and Site ID to list the WLANs

  • Create new GET Request by hitting ‘+
  • Type https://api.mist.com//api/v1/sites/<siteID>/wlans into request URL
    • Ensure to include https:// prefix!
  • Go to ‘Headers‘ tab and specify (type in) Authorization (use Token obtained in previous steps in VALUE fields; ensure to include key word ‘Token ‘ before pasting in the actual token!) and Content-Type (type ‘application/json’ in VALUE field)
  • Hit ‘SEND
O Postman 
File Edit View 
Q Filter 
History 
Help 
Import 
Collections 
Runner 
APIs BETA 
Clear all 
My Workspace • 
Invite 
NO Environment 
GET https•.//api.mist.com//api/vl/sit.., • 
Untitled Request 
(D Save Responses 
Today 
https://api.mist.com//ag 
O GET 98886fb1c48ef.b905-8 + 
4/"' lans 
May 8 
Params 
Authorization 
upgrade 
Save 
Cookies Code 
presets 
Headers (2) 
Authorization 
Content-Type 
Key 
Temporary Headers (7) O 
Body Cookies Headers ( 1 6) 
Headers (9) 
Test Results 
Visualize 
Body Pre-request Script 
Tests 
VALUE 
Token UTEmqoKcpxImnGodssHzozcn8Sk7PCcm. 
application/json 
Value 
status: 2000K 
DESCRIPTION 
Description 
Time: 673ms 
Bulk Edit 
Size: 8.45 KB 
Save Response 
Pretty Raw 
Preview 
"class": "best effort", 
"over"Arite": false 
"org_id": "d2b94bf9-22be-4af1-aød1-68e162fbe199", 
"dtim": 2, 
"hide 55id": false, 
"acct_servers": t], 
"created time": 1557492085, 
"disable_uapsd": false, 
"allow mdns": false, 
"apply_to"• "site", 
"app_limit": { 
"enabled": false, 
"apps" • 
"wxtag_id5": O 
: "a88d6cdd-6aaS-4eb2-8cd9-fb8731ab18e2" , 
"vlan id": 20, 
"wxtag_ids": null, 
"SSI t" 
v an_poo Ing : 
"wlan limit dcmn": 2øøøe, 
"wxtunnel id": null, 
"auth serwers nas id": " 
false, 
"site id • 
". "efc98886-fb1c-48ef-b9ø5-81920edS7dS4", 
"disable Hmm": false, 
"airwatch": { 
"username" . 
"api_key"• " , 
"console url": " 
"password" • " 
"enabled": false 
"schedule": {

Note the “id” of “WiFiNinjas(.)(.)” WLAN

Copy WLAN “id” to our GET URL

*jstman 
File Edit View 
Import 
Runner 
APIs 
Clear all 
My Workspace • 
Invite 
No Environment 
History 
Collections 
GET httpswapi.mist.com"api/vl/sit... • 
Untitled Request 
GET httpswapi.mist.com"api/vl/sit... • 
Save Responses 
Today 
httpswapi.mist.com.'/api/vl 'sites,'0fc98886-fb 1 c-48ef-b905-819200d57d54/wIan . 
Send 
https://api.mist.com//api/vl 'sites/Ofc 
c-48ef-b905-819200d57d5 
4/wlans 
https://api.mist.com//api/vl 'sites/Ofc 
988864b 1 c-48ef-b905-819200d57d5 
4/wlans/a88d6cdd-6aa54eb2-8cd9-fb 
https://api.mist.com//api/vl 'sites/Ofc 
98886-fb 1 c-48ef-b905-819200d57d5 
4/wlans/a88d6cdd-6aa54eb2-8cd9-fb 
https://api.mist.com//api/vl 'sites/Ofc 
98886-fb 1 c-48ef-b905-819200d57d5 
4,'wlans/a88d6cdd-6aa54eb2-8cd9-fb 
https://api.mist.com//api/vl 'sites/Ofc 
98886-fb 1 c48ef-b905-819200d57d5 
4,'wlans/a88d6cdd-6aa5-4eb2-8cd9-fb 
https://api.mist.com//api/vl 'sites/Ofc 
1 9200d57d5 
4,'wlans/a88d6cdd-6aa5-4eb2-8cd9-fb 
https://api.mist.com//api/vl 'sites/Ofc 
1 c48ef-b905-819200d57d5 
4,'wlans/a88d6cdd-6aa5-4eb2-8cd9-fb 
https://api.mist.com//api/vl 'sites/Ofc 
1 c48ef-b905-819200d57d5 
4/wlans 
params 
Authorization 
Headers (9) 
Test Results 
Visualize 
Body 
pre-request Script 
VALUE 
Tests 
Code 
GET 
GET 
PUT 
Settings 
Save 
Cookies 
Presets 
Headers (2) 
Authorization 
Content-Type 
Key 
Temporary Headers (7) O 
Body Cookies Headers ( 1 6) 
Bulk Edit 
Token UTEmqoKcpxlmnGodssHZOZCn8Sk7PC 
application/json 
Value 
status: 200 0K 
Description 
Time: 176ms 
size: 8.5 KB 
Save Response 
P retty 
Raw Preview 
Yesterday 
May 8 
"template åd": null, 
leøøe 
"90s": { 
"class": "best effor•t" 
' false 
"org_åd 
"d2b94bf9-22be-aaf1-aød1-68e162fbe199", 
"dtim": 2, 
"hide ssid": false, 
"acct serwers : 
"created tine . 
"disable_uapsd" 
1557492085, 
. false, 
"allow mdns": false, 
"apply _ to": "site" , 
"app_limit": { 
• 'enabled": false, 
• 'apps": 
"wxtag_ids 
a88d6cdd-6aa5-1eb2-8cd9-fb8731ab18e2

We should now see the details of just “WiFiNinjas(.)(.)” WLAN.

PS. We love that (.)(.) suffix!

Change WLAN name

Let’s rename “WiFiNinjas(.)(.)” to “WiFiNinjasTitties”! 🙂

Since we’re changing WLAN config, we’ll use PUT this time.

Change request type to PUT, go to “Body” tab, ensure “JSON” type is selected and write down lines that you want changed between curly brackets.

Note, that there is no comma at the end of the line.

You can now refresh your Mist Dash or GET the WLAN info via API to check if the name was changed.

Worked like charm! 🙂

What’s Next?

Finally, we can use Postman to translate our actions into code (we’ll stick to Python), that can be used in scripts.

That’s it!

We have successfully listed all Mist WLANs, then narrowed it down to one WLAN and finally we’ve changed the name of that WLAN. All with REST API GET and PUT requests – how cool!

With love,

WiFi Ninjas x

WN Blog 015 – Cisco Catalyst 9800 – Software Upgrade

Upgrading Cisco Catalyst 9800 software is easy and sleek.

You could use all standards protocols to put a new image on the controller: FTP, TFTP, SFTP or HTTPS.

Here are the steps:

  • Download new .bin image from Cisco; I went for the newest TAC recommended image.
  • On the WLC, go to Administration > Software Upgrade or simply start writing ‘upgrade’ in the search bar. Select Transport Type (I used HTTPS), bootflash File System and select newly downloaded image with .bin extension. Click ‘Download & Install‘ and confirm that you really, truly and purposefully want to do it!
  • Wait for the file to finish downloading to the WLC and hit ‘Save Configuration & Reload‘ once Status is all green.
  • Lastly, train your Jedi skills. Be patient.

Installation and reload is extremely fast and takes just a minute or two (at least for the CL version).

Enjoy!

WN Blog 014 – Cisco Catalyst 9800 – Configuration Guide (FlexConnect)

It’s simple, right? Sure! Took me a while to figure that one out.

Our goal in this post is to demo Cisco Catalyst 9800 WLC FlexConnect Configuration.

It’s assumed you’re familiar with all C9800 solution building blocks (we’ve covered it before here) but if it’s your first time, here is very quick recap:

Basic C9800 Configuration Blocks

And this is the lab. Note that VLAN 20 is now removed from the ESXi Trunk on the switch port G0/7. It is no longer needed as the AP plugged to port G0/1 will be dropping users’ data locally now.

Lab Environment

Pre-reqs

  1. Since we’re leveraging FlexConnect local switching (AP puts wireless users into the network, data traffic is no longer tunneled back to the C9800 WLC), AP trunk must allow vlan 20 (that is a wireless users VLAN, local to the AP)
  2. C9800-CL VM is freshly deployed as shown here or it is configured for central switching as shown here
  3. C9800 can communicate with the network; wireless management interface (VLAN 11 in this example) is up
  4. AP is registered to the C9800

In this example, we still have my AP registered as ‘local’ (central switching), centrally switched SSID is up, my phone is associated and has full access following the ‘central switching deployment’ blog here.

AP joined as ‘local’
9800-PSK-Central SSID is up and 1 client (my phone) is associated on 5GHz

The only places where the config is different between Central and Flex are:

  1. Policy Profile – sets SSID set to local switching and maps to a local VLAN
  2. Flex Profile – defines AP Flex attributes like AP Native VLAN, Local Auth and AP Local VLANs are specified here
  3. Site Tag – tells the AP to join as Flex and use specific Flex Profile

I’ll put more wording around the above only, as we’ve already covered all other relevant details in the ‘centrally switched’ blog post here.

Steps

This is how we registered AP as Flex and configured locally switched Flex WLAN.

1. Clean up the config

For simplicity, I just deleted all Profiles and Tags except of RF Profile and RF Tag (and that’s it, I didn’t delete anything else; still, don’t worry if you start with a fresh blank config :))

2. Create new WLAN profile

WLAN Profile – General Tab
WLAN Profile – Security Tab

3. Create Policy Profile

Policy Profile – General Tab

Central Switching” must be unticked to enable Flex Connect Local Switching; it also makes sense to untick “Central DHCP” as we’re probably happier with DHCP process being handled locally and not via a WLC. I also like to include the VLAN ID that we are mapping this Policy Profile to in the Name or Description, as we might have more Policy Profiles mapping different VLANs for different WLANs and it’s good to know what policy does what just by glancing at its name or description.

Policy Profile – Access Policies Tab

“VLAN/VLAN Group” is where you map WLAN to a VLAN! There is no direct equivalent to that mapping as we know from the AireOS. Please note that if you create a VLAN & name it (either through CLI: (config)# vlan 20; (config-vlan)# name LAB-WIRELESS-USERS or GUI: Configuration > Layer2 > VLAN) and use VLAN name to refer to it in a Policy Profile, it WILL NOT WORK! You must refer to a VLAN via its ID (and not a name, since it doesn’t exist on the AP!). If you want to refer a VLAN name here, you must specify 100% matching VLAN ID and corresponding VLAN name in the Flex Profile. See “Flex Profile” section below for more details.

4. Create Policy Tag

Policy Tag, stiching WLAN Profile and Policy Profile together

5. Create AP Join Profile

AP Join Profile – General Tab

6. Create Flex Profile

Flex Profile – General Tab
Flex Profile – Local Authentication Tab
Flex Profile – Policy ACL Tab
Flex Profile – VLAN Tab

We didn’t have to create Flex Profile for Centrally Switched WLAN, but we will need it here. We can use Flex Profile for many different things, but those are quite important:

  • General Tab
    • Native VLAN ID – this is where we specify AP mgmt. VLAN that the AP will be sitting in.
    • Efficient Image Upgrade – this means that when we upgrade the controller code, it will be pushed to just one Flex AP (called the FlexConnect Master AP in AireOS) tagged with a Site Tag containing the same Flex Profile. The code will then be distributed to the remaining APs locally without the need to transfer it over WAN or Internet multiple times. Neat.
  • Local Authentication Tab
    • This is where we can specify a RADIUS server local to the AP for wireless clients authentication so it doesn’t have to be central and go through a DC somewhere far far away. Radius Server Group can also be used in a very valid scenario, where the preference is to use central RADIUS for authentication and visibility and switch to local (from AP perspective) RADIUS when the central one is down. EAP based WLANs would gain a survivalability element in case the WLC or communication to the WLC goes down. Existing clients could potentially re-authenticate and new ones could connect but bear in mind that Flex AP and WLAN in not-connected mode (WLC not reachable) would lose access to RRM and roaming optimisation mechanisms.
  • Policy ACL Tab
    • If you ever want to use any ACLs on APs / WLANs configured as Flex Local Switching, you must create an ACL in ‘Configuration > ACL’ and, on top of that, you MUST add this ACL under ‘Flex Profile > Policy ACL’ tab! By doing so, the specific ACL is pushed down to the AP and can be refered to (statically or via RADIUS).
  • VLAN Tab
    • This is where you can create VLANs that will be pushed down to the AP. Remember VLAN/VLAN Group under Access Policies Tab of the Policy Profile, where we mapped the profile to wireless users’ VLAN 20 using VLAN ID? If we’d like to refer to the VLAN by its name, we would need to have a matching VLAN/name configured here!

7. Create Site Tag

Site Tag

We’ve come to the last place, where Flex relevant config sits! The second we untick “Enable Local Site”, “Flex Profile” dropdown appears. For the AP to join the WLC as a Flex AP, we need to untick “Enable Local Site” and select “Flex Profile” that the AP will use.

8. Create RF Profile (for 2.4 and 5GHz) and RF Tag

Since I created them in our ‘central’ switching blog and didn’t delete them, refer to our blog here to find out more about RF Profiles and Tags.

Apply all tags to relevant APs

AP(s) will now reboot and should join back as a Flex AP and broadcast our SSID:

AP Joined in Flex AP Mode
Flex SSID “9800-PSK” is broadcasted and a client (my phone) is happily connected

That’s it! 🙂 We massively hope it was helpful for someone!

Tons of love,

WiFi Ninjas x

WN Blog 013 – Cisco Catalyst 9800-CL – Redundancy HA SSO (CLI and Deeper Dive)

Hey! And welcome to another C9800 blog 🙂 Recently, Matt has blogged about Redundancy configuration using GUI and covered HA/SSO operation here. Today, I wanted to do a deeper dive into the topic. Enjoy!

Lab Environment:

In my lab, both C9800-CL VMs will sit on a single ESXi box. In the real world, they would rather be distributed across separate VM hosts or even separate geographical locations. In any case, configuration would stay identical and same pre-reqs are still relevant. Most importantly, both vWLCs need access to the LAN on their mgmt. interface (that will also be used by APs to register to) and a separate ‘P2P’ L2 link to form & monitor HA and sync configuration.

See a full picture of my lab below:

Lab – Fuill Picture

And to make things simple, also see a simplified logical diagram of C9800-CL networking and interfaces mapping to the VMware vSwitches:

Lab – Simplified Diagram

HA/SSO nitty-gritty stuff

  • HA/SSO in C9800 works differently to how it works in Aironet, it’s more like a VSS now.
  • Initially, both C9800-CL WLCs will require management IP address.
    • Note, that management IP is optional for a secondary WLC if you have a console access to it. Mgmt IP is no longer required to form a HA Pair
  • Redundancy Management as we know it from AireOS (additional IP per WLC from the same subnet as mgmt.) doesn’t exist anymore and therefore Redundancy Port IP is no longer automatically derived from the Redundancy Management IP (it used to be 169.254. + last two octets of the Redundancy Management IP) – you have to manually specify “Redundancy Port” IP and elect one of the VM interfaces to be used as a Redundancy Port (RP)
  • “Redundancy Port IP” is now called “Redundancy Local IP” (GUI) or “Local HA Interface IP” (CLI) <- same thing, just called differently in different places
    • Local HA Interface IP (I’ll call it just that as we’ll use CLI to form HA/SSO in this example) should be in a different subnet to a management interface. This subnet should not be routable.
  • Both WLCs are using the same mgmt. IP to maintain AP CAPWAP during failover once HA/SSO is formed
  • Two units are using a dedicated Redundancy Port (now called HA-Interface) to sync
  • Role election happens at boot, uses priority (1-15, default is 1; higher priority is preferred) or lowest MAC if priority is the same; two controllers are supported, not more
    • It’s best practice to assign higher priority to the preferred active WLC
  • Standby WLC continuously monitors Active via RP keepalives
  • Gratuitous ARP is sent by a Secondary WLC that is taking over the Active role
  • No preempt functionality is supported – failback, if needed, is manual
  • Timers:
    • SSO Failure Detection: 50 ms
    • Reconciliation Time (Standby becoming Active): max 1020ms
  • Reloading Active WLC via CLI: # reload will also reload a standby immediately
  • Forming HA wipes out the config!!!
    • Always form a HA first, configure later
  • WLC Interface used for HA will disappear from the interfaces list (sh ip int br etc. will not list it anymore)

HA/SSO Pre-reqs

  • HA Pair can only be formed between two WLCs of the same form factor (C9800-CL VMs in this example)
  • Both WLCs must be running same code version
  • Max RP RTT = 80ms, min bandwidth = 60Mbps, minimum MTU = 1500
  • Both WLCs are built, accessible, VMware networking is configured, WLCs have access to the LAN and separate L2 link for the HA is available between the WLCs
    • C9800-CL HA-Interface that we intend to use as a Redundancy Port L2 HA inter-vWLC link must be put into a separate, unused VLAN! Using existing VLAN/subnet, especially if it’s management VLAN/subnet, would theoretically work (I’ve tested it) but it will cause instability, duplicate ARP entries and ARP resolution issues. See below the ‘show logging’ output from my switch, where I had HA-Interfaces IP sitting in the mgmt. subnet:
  • See full deployment guide of the C9800-CL for VMware ESXi in our previous blog here
  • Prepare IP addresses and subnets; in this example we will use the following:
    • Active WLC
      • Chassis 1, priority 2
      • Mgmt. (VLAN 11): 10.10.11.35 /24
      • HA Local IP (VLAN 666): 10.10.66.35 /24 (VLAN 666
    • Backup WLC
      • Chassis 2, priority 1
      • Initial Mgmt (VLAN 11): 10.10.11.40 /24 (optional if have console access, won’t be used anymore once HA is formed)
      • Mgmt: (VLAN 11) 10.10.11.35 /24 (shared between HA WLCs once HA is formed)
      • HA Local IP (VLAN 666): 10.10.66.40 /24

HA/SSO Configuration

I feel CLI gives us better visibility into what is happening on both WLCs while forming HA Redundancy, therefore we will use CLI here. It is possible to use GUI too, but I am not a huge fan of “click, wait, pray and hope for the best” approach, where I can’t see what’s happening 🙂

CLI

  • Active C9800-CL
    • # chassis ha-interface Gig 3 local-ip 10.10.66.35 /24 remote-ip  10.10.66.40
    • # chassis 1 priority 2
    • # wr
    • # reload
  • Standby C9800-CL
    • # chassis ha-interface Gig 3 local-ip 10.10.66.40 /24 remote-ip  10.10.66.35
    • # chassis 1 renumber 2
    • # chassis 1 priority 1
    • # wr
    • # reload

GUI

Refer to our previous C9800-CL Redundancy blog written by Matt here

Note: Standby WLC would need to be accessible on its management IP address to use GUI for Redundancy configuration.

Tshoot & Validation

It is good and useful to know how to monitor, tshoot and validate HA Redundancy configuration. You can do it in both CLI (see below) and GUI (Monitoring > System > Redundancy; Dashboard > Slot: Active / Standby)

Chassis

# show chassis

Show Chassis
  • Validate Chassis# and Priority. Also see the HA state (“Ready” is the final and desired stage, you can also see “Initializing” etc.) and HA-Local IP of both C9800-CL VMs.

# show chassis ha-status local

Show Chassis HA-Status Local
  • Check before the reboot while forming the HA
  • Active should have higher priority, HA Local and Remote IPs and assigned interfaces are shown, together with Chassis# and Priority that will be assigned to this local WLC upon reboot (you might have changed those values and it can be slightly confusing, so it’s best to check after configuring and before rebooting_

Redundancy

# show redundancy

Show Redundancy
  • Another useful show command, where we can validate uptime in current state, versions, modes and more!

Want to Know More?

Yeh, totally get it – me too 😉 I’ll drop some useful stuff that I came across and used at some point here that might help with configuration and troubleshooting.

Manual failback

  • # redundancy force-switchover
  • Can be very useful in scenarios like primary DC maintenance, where Backup C9800-CL takes over. Since there is no preempt mechanism in C9800 HA (former Active WLC won’t claim its original Active role back when it comes back online)

Clear HA Configuration

  • # chassis clear
  • # reload

This is how it looks in the CLI:

Note: it’s enough to clear chassis information on the active HA Pair WLC, despite the information that the ‘other’ (Backup) chassis will keep its HA configuration. I’ve tested it and it will not! Backup chassis will also reboot with clear redundancy config!

  • Chassis clear is used to break the HA; be careful with this as you will end up with duplicate IPs! Both former Active and Passive WLCs will share identical configuration upon reboot!

Duplicate IP addresses after clearing chassis config

Successful HA/SSO Formation

This is how successful HA  formation should look like in a console of both Active and Passive WLCs:

Successful HA/SSO Formation upon reboot once ha-interface was configured; Active (left) was elected Active due to higher priority

That’s it for now! It feels great to play around with new toys and discovering differences between old and new WLCs! Don’t hesitate to comment, add to the discussion or simply let the world know that you are enjoying beta-testing Cisco gen 1 products as much as we are 😉

WN Blog 012 – Can You Crack 802.1X WPA2-Enterprise Wireless Data?

One of our clients has recently approached me with concerns about their new WiFi network that we were planning to put in. They were coming from a wired-only environment and were not sure if introducing EAP-TLS based corporate wireless was a good and safe idea. Additionally, while preparing for my CWAP exam I heard in one of the course videos that “you can’t decrypt 802.1x EAP, as there is no known key that we can enter to start 4-way handshake”. But is it really the case?

I will answer this question by first touching on the 802.1X EAP authentication framework recap that will help us understand conditions that must be met, and steps taken to decrypt WPA2-Enterprise data.

All modern EAP variations are using strong CCMP encryption. Instead of attacking it, we will focus on capturing RADIUS packets on the wire and extract a PMK from this transaction. We will then capture 4-way handshake to get Anonce and Snonce and use it together with PMK, Supplicant MAC and Authenticator MAC to derive PTK (Wireshark can do it for us) used to decrypt our wireless session.

802.1X EAP Recap

IEEE 802.1X is a standard for Network Access Control. It provides authentication (making sure that something is what it claims to be) mechanism to devices wishing to connect to a LAN or WLAN. 802.1X defines the encapsulation of the Extensive Authentication Protocol (EAP) over IEEE 802, that is known as EAP over LAN (EAPOL). 802.1X defines 3 roles: Supplicant (client), Authenticator/NAS (AP) and Authentication Server (RADIUS). Successful EAP transaction starts a process of 802.11 Security Keys Generation, that I tried to visualise in a diagram below, together with 802.11 Open Authentication, 802.11 Association, Tunnelled EAP Authentication and a 4-Way Handshake, collectively being part of an 802.1X standard.                   

802.1X EAP and 802.11 Security Keys Generation Process
802.1X EAP and 802.11 Security Keys Generation Process

Conditions

The following conditions must be met to decrypt 802.1X EAP encrypted captures:

1. RADIUS key must be known

  • Brute force against the RADIUS capture is an option but strong RADIUS key would make it unpractical. Make sure the key is strong!
  • Social engineering attack – get network engineer’s contact details and ask him/her about a RADIUS key saying you’re from NOC tshooting an issue or a contractor working on upgrading RADIUS server and see what happens. Make sure staff is trained how to handle social engineering attacks and that their contact details like phones, mails and positions within the company are well secured!
  • Access to RADIUS server – some mainstream RADIUS servers (MS NPS, FreeRADIUS) store the key unencrypted in a file. Cisco ISE can show you password when you’re logged in. Make sure access to the RADIUS servers and logon credentials are properly secured!

2. Wireless capture of the session that we want to decrypt must be taken

  • Session must include 4-way handshake, so must include both packets coming from the client and AP, meaning that the potential attacker would need to be physically close to both. 

3. Wired capture of RADIUS authentication must be taken

  • Capture of RADIUS authentication on the wire is essential, as PMK is never sent over the wireless, so it can’t be eavesdropped. We’ll extract it from RADIUS wired captures.
  • Capturing RADIUS traffic would require physical access to the LAN to plug a collector into or access to the network infra to configure SPAN. Make sure that the admin access to the network equipment is secure and that all infrastructure is physically locked!

Lab Environment

Here is the lab setup and capture locations (both wired and wireless):

  • Wireless Captures: Cisco AP in sniffer mode placed between my wireless test client and client serving AP, configured to send captures to my Windows Server VM running Wireshark.
  • Wired Captures: Cisco switch with SPAN monitoring session and port facing my ESXi server (with Cisco ISE RADIUS VM running there) being a SPAN source and switch port facing my laptop being a SPAN destination.
Lab Network Diagram
Lab Network Diagram

Steps

Here is a high-level summary of what we need to do to decrypt our WPA2-Enterprise wireless session:

  • Extract PMK with wired RADIUS captures; use RADIUS Shared Secret, Request Authenticator from the final Access-Request RADIUS frame and MS-MPPE-Recv-Key from the RADIUS Access-Accept frame.
  • Capture 4-Way Handshake with wireless captures.
  • Use extracted PMK and 4-Way Handshake to derive PTK with Wireshark and use it to decrypt user data.

Note: PTK is valid only for the duration of a single session. Session timeout, new association or re-association (roaming) would require to derive new PTK!

Assuming all conditions are met, let’s crack on!

1. Obtain RADIUS key

  • Since I’m using ISE, here is where I would look to get it:
RADIUS key configured in Cisco ISE
RADIUS key configured in Cisco ISE

2. Start capturing wireless traffic of interest

  • Position capturing device so it can capture both wireless client and AP traffic.

3. Start capturing wired RADIUS traffic between the Authentication Server and the Authenticator

  • Ensure to capture full RADIUS exchange for the wireless device authentication.

4. Connect wireless device to the EAP SSID

  • If it’s your test device, disconnect and then connect again.
  • If it’s not your test device, you could try to force the device to re-connect by sending a de-auth; SSID must not use management frames protection mechanism, and they usually don’t for compatibility reasons.

5. Obtain Authenticator from the last Access-Request packet in wired RADIUS capture

  • Go to RADIUS Protocol > Authenticator.
  • Here it’s 00:0d:42:73:f6:19:5c:d3:88:73:cf:b3:2c:76:5d:16 (you can copy value in hex straight from the capture).
Authenticator value from Access-Request wired RADIUS capture
Authenticator value from Access-Request wired RADIUS capture

6. Obtain MS-MPPE-Recv-Key from the Access-Accept packet in wired RADIUS capture

  • Go to RADIUS Protocol > Attribute Value Pairs > AVP: t=Vendor Specific (last one)
  • Here it’s cf:6f:b5:06:da:57:b1:9c:e4:6d:76:af:93:51:59:7e:2c:f8:cd:79:c6:2b:e1:a5:4f:ab:28:bd:ed:d3:81:d3:a9:57:dd:74:f8:d1:41:b8:ec:50:ea:d7:27:75:85:d3:1e:d3
MS-MPPE-Recv-Key value from Access-Accept wired RADIUS capture
MS-MPPE-Recv-Key value from Access-Accept wired RADIUS capture

7. Compile PMKextract code, that we will use to extract PMK

  • I used Visual Studio in Windows to build the code mentioned earlier. There are also Python versions of similar code available but since I’ve already had Visual and it worked, I focused on this approach.
  • Create a new C++ project/file in Visual and paste this code:
#include <stdio.h>
#include <stdlib.h>
#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#define _WIN32_WINNT 0x0400
#include <wincrypt.h>

typedef unsigned int u32;
typedef unsigned char u8;

//
// This debugging function can be found in wpa_debug.c from the hostap package
//
//extern void wpa_hexdump_key(int level, const char *title, const u8 *buf, size_t len);

#define os_malloc(s) malloc((s))
#define os_free(p) free((p))
#define os_memcpy(d, s, n) memcpy((d), (s), (n))
#define MD5_MAC_LEN 16

static void cryptoapi_report_error(const char *msg)
{
 char *s, *pos;
 DWORD err = GetLastError();

 if (FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER |
     FORMAT_MESSAGE_FROM_SYSTEM,
     NULL, err, 0, (LPTSTR) &s, 0, NULL) == 0) {
   printf("CryptoAPI: %s: %d", msg, (int) err);
 }

 pos = s;
 while (*pos) {
  if (*pos == '\n' || *pos == '\r') {
   *pos = '\0';
   break;
  }
  pos++;
 }

 printf("CryptoAPI: %s: %d: (%s)", msg, (int) err, s);
 LocalFree(s);
}

int cryptoapi_hash_vector(ALG_ID alg, size_t hash_len, size_t num_elem,
     const u8 *addr[], const size_t *len, u8 *mac)
{
 HCRYPTPROV prov;
 HCRYPTHASH hash;
 size_t i;
 DWORD hlen;
 int ret = 0;

 if (!CryptAcquireContext(&prov, NULL, NULL, PROV_RSA_FULL, 0)) {
  cryptoapi_report_error("CryptAcquireContext");
  return -1;
 }

 if (!CryptCreateHash(prov, alg, 0, 0, &hash)) {
  cryptoapi_report_error("CryptCreateHash");
  CryptReleaseContext(prov, 0);
  return -1;
 }

 for (i = 0; i < num_elem; i++) {
  if (!CryptHashData(hash, (BYTE *) addr[i], len[i], 0)) {
   cryptoapi_report_error("CryptHashData");
   CryptDestroyHash(hash);
   CryptReleaseContext(prov, 0);
  }
 }

 hlen = hash_len;
 if (!CryptGetHashParam(hash, HP_HASHVAL, mac, &hlen, 0)) {
  cryptoapi_report_error("CryptGetHashParam");
  ret = -1;
 }

 CryptDestroyHash(hash);
 CryptReleaseContext(prov, 0);

 return ret;
}

int md5_vector(size_t num_elem, const u8 *addr[], const size_t *len, u8 *mac)
{
 return cryptoapi_hash_vector(CALG_MD5, 16, num_elem, addr, len, mac);
}

static u8 * decrypt_ms_key(const u8 *key, size_t len,
      const u8 *req_authenticator,
      const u8 *secret, size_t secret_len, size_t *reslen)
{
 u8 *plain, *ppos, *res;
 const u8 *pos;
 size_t left, plen;
 u8 hash[MD5_MAC_LEN];
 int i, first = 1;
 const u8 *addr[3];
 size_t elen[3];

//wpa_hexdump_key(1, "key", key, len);
//wpa_hexdump_key(1, "secret", key, len);
//wpa_hexdump_key(1, "auth", req_authenticator, MD5_MAC_LEN);

 /* key: 16-bit salt followed by encrypted key info */

 if (len < 2 + 16)
  return NULL;

 pos = key + 2;
 left = len - 2;
 if (left % 16) {
  printf("Invalid ms key len %lu\n", (unsigned long) left);
  return NULL;
 }

 plen = left;
 ppos = plain = (u8*)os_malloc(plen);
 if (plain == NULL)
  return NULL;
 plain[0] = 0;

 while (left > 0) {
  addr[0] = secret;
  elen[0] = secret_len;
  if (first) {
   addr[1] = req_authenticator;
   elen[1] = MD5_MAC_LEN;
   addr[2] = key;
   elen[2] = 2; /* Salt */
  } else {
   addr[1] = pos - MD5_MAC_LEN;
   elen[1] = MD5_MAC_LEN;
  }
  md5_vector(first ? 3 : 2, addr, elen, hash);
  first = 0;

  for (i = 0; i < MD5_MAC_LEN; i++)
   *ppos++ = *pos++ ^ hash[i];
  left -= MD5_MAC_LEN;
 }

 if (plain[0] == 0 || plain[0] > plen - 1) {
  printf("Failed to decrypt MPPE key\n");
  os_free(plain);
  return NULL;
 }

 res = (u8*)os_malloc(plain[0]);
 if (res == NULL) {
  os_free(plain);
  return NULL;
 }
 os_memcpy(res, plain + 1, plain[0]);
 if (reslen)
  *reslen = plain[0];
 os_free(plain);
 return res;
}

void processTokens(char*  authenticator,
       u8*   processedAuthenticator,
       char*  recvKey,
       u8*   processedRecvKey )
{
 // Handle authenticator
 char* ptr = strtok( authenticator, ":");
 int i = 0;
 while( ptr )
 {
  processedAuthenticator[i++] = ( u8 ) strtoul( ptr, NULL, 16 );
  ptr = strtok( NULL, ":");
 }

 // Handle key
 ptr = strtok( recvKey, ":");
 i = 0;
 while( ptr )
 {
  processedRecvKey[i++] = ( u8 ) strtoul( ptr, NULL, 16 );
  ptr = strtok( NULL, ":");
 }
}

void dumpPmk( const u8*  pmk )
{
 printf( "PMK is:\n" );
 for( int i = 0; i < 32; i++ )
  printf( "%02x", pmk[i] );
 printf( "\n" );
}

int main(int argc, char*argv[])
{
 if( argc != 4 )
 {
  printf( "Usage: %s secret authenticator recv-key", argv[0] );
  return( 1 );
 }

 if( strlen( argv[2] ) != 47 )
 {
  printf( "Bad authenticator length" );
  return( 1 );
 }

 if( strlen( argv[3] ) != 149 )
 {
  printf( "Bad recv-key length" );
  return( 1 );
 }

 u8 processedAuthenticator[16];
 u8 processedRecvKey[50];
 u8* pmk;
 u32 pmklen = 0;

 processTokens(argv[2], processedAuthenticator, argv[3], processedRecvKey );

 pmk = decrypt_ms_key(processedRecvKey, 50,
      processedAuthenticator,
      (u8*)argv[1], strlen(argv[1]), &pmklen);

 dumpPmk( pmk );

 os_free(pmk);

 return(1);
}
  • Save .cpp file. I saved it as “PMKextract.cpp”:
 Adding PMKextract.cpp to the Solution Explorer
Adding PMKextract.cpp to the Solution Explorer
  • If you try to build it now, you’d probably see the error:
Unsafe function or variable warning
Unsafe function or variable warning
  • To allow project to build successfully, add _CRT_SECURE_NO_WARNINGS under Project -> Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions:
Allow project built by modifying Preprocessor Definitions
Allow project built by modifying Preprocessor Definitions
  • Build > Build project should now be successful:
Successful build of the PMK extracting code
Successful build of the PMK extracting code

I then copied compiled ‘Project1.exe’ file to C:\Geekwifi and renamed it to ‘PMKextract.exe’.

8. Extract PMK

  • Navigate to the location of your PMKextract.exe file. It takes all required attributes as shown:
PMKextract usage
PMKextract usage
  • Get the PMK:
Extracting PMK using all values that we gathered: RADIUS secret, authenticator and Recv-Key
Extracting PMK using all values that we gathered: RADIUS secret, authenticator and Recv-Key

9. Specify decryption key in wireless captures in Wireshark

  • Finally, open wireless captures and use our extracted PMK as a wpa-psk key.
Provide Wireshark with extracted PMK in ‘Edit > Preferences > Protocols > IEEE 802.11 > Decryption keys > Edit’
Provide Wireshark with extracted PMK in ‘Edit > Preferences > Protocols > IEEE 802.11 > Decryption keys > Edit’

10. Enjoy decrypted wireless captures!

  • We can see all the usual stuff – probes, authentication and association requests and responses, EAP process with EAP Success at the end, 4-way handshake and then decrypted data. Happy days!
Decrypted 802.1X WPA2-Enterprise session
Decrypted 802.1X WPA2-Enterprise session

Conclusion

Properly configured and physically secured WPA2-Enterptise wireless network, especially where client and server certs are involved, is still considered highly secure. Specific conditions must be met to decrypt 802.1X EAP wireless session captures, where RADIUS key must be known, and the attacker would have to be able to capture RADIUS conversation on the wire and 4-Way Handshake on wireless to make it possible. Additionally, decrypting WPA2-Enterprise session does not necessarily mean we could eavesdrop on the meaningful user data, as it might be encrypted on a data level, i.e. using TLS or SSL, that do not rely on WiFi infrastructure encryption.

Literature