Friday, June 18, 2010

Chapter 8 – Half of Something is Better than Nothing, but Why Settle?

Our network is pretty inexpensive but suffers from first generation bandwidth problems of losing ½ the bandwidth capacity every hop. Patton wouldn’t settle for a partial victory and neither will we since we can do it for very little additional cost. It’s possible to virtually eliminate bandwidth loss per AP for up to $400 for the Gateway points ($100 for every direction it has to backhaul) $200 for midpoints and $100 for the end point. Keep in mind you don’t need to do this at every AP, just the links that have a higher load or to extend the network further. This is also something that can easily be added later as capacity increases.
We currently have the AP cost at about $200 and each AP can handle 30-50Mbps of traffic. However, 2 hops down from the gateway point or backhaul point, we are now down to about ½ of that. Go another hop and its ¼ of that using single radio WDS functionality. WDS has 3 other problems that we can’t get around with Ubiquiti’s Bullet 2M HP’s. The first is that we lose the client isolation function. The second is we lose a proprietary function that we will later cover called AirMax. The third is that we lose advance WPA encryption methods and we are limited to the evermore useless WEP. It’s easy to get around this though. Keep in mind the motto, “it’s not a problem if it can be fixed with money”.
Ubiquiti sells a little radio called a Nanostation Loco M5. It’s available in both 2.4GHz and 5.8GHz versions. We will focus on the 5.8GHz version since that limits the interference for our backhaul and as we discuss later, opens up new options for our network for PTMP. This little fellow only costs $70 and provides a 2x2 MIMO signal that can support up a total of around 100Mbps of total throughput with a 20MHz channel. I’m going a little conservative on throughput numbers since different types of IP traffic can test between 60-150Mbps. Put 2 of them on the pole with the Bullet M2 and your backhaul can now support all that bandwidth across several hops with a loss of about 1Mbps and about 2-3ms of latency added on each hop. However, you have just added another $200 per AP to do this with a switch. If you have two 5.8GHz radios on the pole along with the Bullet, you will need a switch.
The most cost effective switch I have found that is designed to handle reasonable temperatures is the Linksys SD205 unit. Although they are rated for 122 degrees, I have never seen one fail and I have had them in temperatures far in excess of 140 degrees for years. For around $25, they are the best little units for sealed outdoor installations and there is an SD-208 version if you need more ports. Compared to an industrial switch which starts at $250 (cheapest ones I have seen), the Linksys units are a steal. If you need a managed switch, then you are looking at industrial switches and the cost is going to start around $450.
The other good part of adding the Nanostation Loco M5 to our system is now if we have to add more AP’s in the middle of our network, we don’t have worry about the 1/n issue. We can also use WPA or AES security on the 2.4GHz network and 5.8GHz backhaul hops. This solves the wireless security issue. If we have to go around corners or Y off an AP, we can add up to 4 Nanostation Loco M5’s per pole if we use a 20MHz wide channel. Yes, I know that theoretically you can do 5 channels but that leaves no buffer between channels. With 4 channels, you get a 5MHz buffer which although not ideal, it’s not bad.
The drawback to the Nanostation Loco M5 is that to get maximum throughput or MCS (15) rates, you will be limited to 17dBm output with a 13dBm antenna or a total of 30dBm EIRP. If there are any tree obstructions or you need more power, the big brother, the Nanostation M5 has a 16dbi antenna and 21dBm of output for an extra $20. Keep in mind that these radios will go to a much higher power level if necessary but it comes at a reduction in throughput which we discussed in article 2. It does however, leave some room for increasing power if interference or obstructions start to cause issues later.
The Nanostation Loco M5 radios can also do one other thing. If we get back to the idea of a PTMP hybrid system, that means that users that are within the beam pattern of the Nanostation Loco M5’s can actually connect directly to the 5.8GHz backhaul. I suggest this stay reserved for truck rolls and trained technicians. 5.8GHz is a lot more sensitive to LOS which makes it more difficult for clients to install themselves. We are going to extend this concept further later also.
If the goal is that clients are going to install their own CPE’s, either indoors or out, it’s best if clients use a 2.4GHz product, preferably a Nanostation 2M or the upcoming Nanostation Loco 2M for indoor use with a window/wall mount. There is also some new indoor equipment from Ubiquiti, the WiFiStation, with directional and omni-directional antennas that support 802.11N and only cost around $30. The range is obviously lower but it also lowers the cost for clients.
Although our network also doesn’t support 2x2 MIMO on 2.4GHz yet, it is now supporting it on 5.8GHz so things are moving along. We have a couple options coming upon the 2.4GHz 2x2 MIMO solution coming up. There is also a way to customize the firmware on these units so that the CPE can be fixed only on your network with the proper settings and would allow you access to manage settings on the client side to optimize their connection quality. Imagine the client connecting to the system and then you get to remotely upgrade firmware on every client device simultaneously, monitor their connection quality, and manage the connection all way to the computer. That is the advantage to having all the radios from a single vendor. Tech support calls would go way down and staff wouldn’t have to fudge their way through 100 different products. Taking this a step further, if all the devices were Ubiquiti M series radios, you could also use the AirMax feature with polling for the clients. Of course, you sort of kiss off the legacy devices, or do you?

Tuesday, June 15, 2010

Chapter 7 – One Size Does Not Fit All

Before we continue developing the system further, I think it’s a good idea to discuss all the various antenna designs that go into a municipal design process. I’ve described one type of design with omni-directional antennas, although I use many difference designs customized to the target client. It won’t meet all needs; no system will unless the budget is unlimited. It is being designed to be as flexible as possible, but there are specific technologies that may work better in some areas. Wireless hardware manufacturers have put forth various designs and optimized their hardware towards that goal. Budgets have sometimes dictated other designs such as “Tales from the Towers.” Due to new equipment that has been released, the numbers of potential designs that can be deployed have exploded.
Let’s start with the legacy systems that started the whole muni-wireless market. The most basic APs were single radios with a single, or dual, omnidirectional antenna design. These systems covered a fixed area and then simultaneously handled backhaul to the next radio. The first problem with this design was simply the fact bandwidth drops by ½ every hop. Although they also supported diversity for the antennas, that didn’t significantly increase the range. It did reduce fading since 802.11b wasn’t really created with a multipath environment in mind.
Eventually, manufacturers went to 802.11g and started adding 2-3 or more radios with secondary radios for backhaul. Using 5GHz frequencies to hand off the backhaul functions increased user throughput, and with dual 5GHz radios, the multi-hop bandwidth loss problem was effectively eliminated. The multi-radio design also spawned some extremely unique AP designs. Manufacturers started adding directional antennas, multiple frequency coverage, integrated sector antennas, beam-forming, and a few other ideas that don’t pass the smell test for actual performance advantages. However, the reason that there are so many systems out there is that there is an actual need for different features. I have used, and will continue to use, many of these products because of their uniqueness.
Jump forward to post-802.11N MIMO technologies, and the number of options from both technology and a budgetary position are mind-boggling. I typically go through no fewer than 5 designs and multiple products when trying to find the best design for a client. I still have questions on design ideas that I haven’t deployed yet that I’m testing. Since most APs today are multiple radios, the exception being the system we are designing for TriadLand, we are simply going to discuss the 2.4GHz side of the APs.
Because 802.11n is simply faster than b/g, we will stay focused there with the idea of backward compatibility. 1x1 MIMO, 2x2 MIMO, 2x3 MIMO, single-polarity, multi-polarity, and beam-forming are all being deployed. Which one is the best? Actually, most of them have some unique feature. It depends on the application. The right answer is the one that solves the problem within a specific budgetary or financial target. So does that mean there is a universal AP? The short answer is no. The really long answer, which I’m going to need 2 more pages to cover, is still no, but there are ways around most of the issues. Some of the answers, I really don’t know right now because I’m still testing some new ideas. Recent discussions and some new projects have gotten peaked my interest.
Let’s get back to our simplest AP design--our “Tales from the Towers” model. It’s a single omni-directional antenna on a 1x1 single stream 802.11 AP. If all clients are 802.11n compatible, then it can support 30-35 802.11b/g/n clients with a total throughput of about 30-50Mbps, TCP/IP. This assumes good LOS coverage, low-interference, and a low-reflectivity environment. We improved the range by using a very-high gain collinear antenna, which has a higher gain over most of the AP’s that use 6-9dBi antennas. It’s not perfect, but it’s cheap, and sometimes that’s all one needs.
802.11n has a distinct advantage over 802.11b/g, MIMO technology. To take true advantage of it, you need multiple either multiple antennas or multi-polarity dual-feed antennas. The question is, do you use multiple vertical antennas or multiple antennas in different polarities. Do you use 2x2 MIMO, 2x3 MIMO, or 3x3 MIMO? Which one works better with legacy 802.11b/g devices, and does it matter?
To answer that question, you first have to understand antenna polarity. The most commonly used polarities are vertical, horizontal, and circular. We used a vertical polarity omni-directional on our original design, which means vertically polarized in relation to the ground. We did it mainly for budget reasons, but how well it works depends on what the polarity of the client device is.
Let’s examine the typical laptop. Early WiFi-enabled laptops simply had small wires or circuit board antennas embedded on the WiFi card internally. Since the board/wire is typically laid flat in parallel to the table, the antenna would be considered a horizontal polarity antenna. So what happens when a vertical antenna connects to a client with a horizontal polarity antenna? The result is up to a 20dB loss of signal assuming both antennas on both sides are the exact same specification in alternate polarities. In reality, most laptops now have wire antennas that are run up the side of the LCD display and sometimes across the top. That gives it both a vertical and horizontal polarity. This is the exact same antenna design for your AM/FM car radio that is embedded inside the windshield.
Let’s start with the idea of how an antenna creates gain. Antenna gain effectively multiplies the signal being fed into it by borrowing the signal from other directions and refocusing it. For example, a 0dBi antenna is actually theoretical. It’s simply a point in space that radiates an equivalent amount of signal in every direction. Think of the center point of a ball. However, add a driven element and a reflector element, along a horizontal support arm at specific distances and you have a 2-element Yagi antenna that has 6dBi gain in one direction.
We will start with a 0dB antenna. A 0dB gain vertical antenna is really a 2.15dBi gain vertical antenna. That means it transmits 2.15db more power along the ground plane than it does straight up. If we make the antenna longer in multiples of the wavelength, then we get more gain. In reality, the highest non-collinear design I have seen is 12dBi in 2.4GHz. The resulting transmission pattern now gets squashed as less signal radiates upward and more signal gets transmitted along the ground for more range. Antenna theory is still developing with new algorithms coming out not only by engineers and scientists, but also by software programs that are discovering more efficient designs.
So how does 15dBi gain compare to 0dBi? In general, signal doubles in distance for every 6dB of gain. A 3dB signal gain increases the EIRP by a factor of 2. 6dBi gain would increase your EIRP output by 4 times which gives you about twice as much range. 15dBi antenna increases your range roughly by a factor of 6 times.
How does this play out in real life? Keep in mind that there will be obstructions in most areas. That means that getting ¾ of the way through a brick wall isn’t a whole lot more effective than getting ½ of the way through the wall. For walls or obstructions with less attenuation, we discussed how a 15dBi antenna can make penetration through an extra wall a reality due to a 6-8dB increase, or more, over the antennas that most metro APs used. A dual-polarity antenna with a lower gain can produce similar results. I have seen 2.4GHz multi-polarity antennas penetrate better than 900MHz single-polarity radios.
2x2 MIMO provides the option of 2 antennas, both in the same polarity or one horizontal and one vertical. There are even antennas that can do dual polarity or circular polarity in an omni-directional, or directional ,design. There are other variations on MIIMO, such as 2x3, 3x3, or more. If the antennas are directional, polarity is simple and cheap. If the antennas are omni-directional, vertical polarity is still cheap. Horizontally polarized antennas were much more expensive as gain goes up, but recent product releases demonstrate multi-feed dual-polarity antennas have come down significantly. Even dual-polarity parabolic dishes have dropped in price. We will cover these in future articles.
One of the more common horizontally-polarized antennas is the waveguide antenna. In an omni-directional design, they can deliver 13-15dBi, or more, of gain. Directional versions range from 14-18dBi. We used directional wave-guide antennas in some of our installations, and they worked great with 802.11b. One test we did demonstrated a laptop with a Cisco PCMCIA card connecting at 1.2 miles inside a fast-food restaurant.
Another popular design is the circular polarity antenna. The advantage to this antenna is that it transmits in all polarities simultaneously. The disadvantage over a single polarity antenna is that it sacrifices 3dB of gain for that multi-polarity coverage. Most circular polarity antennas are directional, although there are variations such as the Lindenblad design which is omni-directional. All antennas are compromises in terms of gain, direction, design, and cost. That is the reason it’s important to first define the target client before even considering any design idea.
There are a lot of variables to proper design of a system. Although the AP should be the easiest part, not including the antennas, beyond design scope, even firmware of the devices is important. Some APs handle packets differently than others. We discussed CPU overhead and real throughput in earlier articles. Firmware bugs and features also make a huge difference. Now throw in an antenna designs, network management, authentication, security, terrain, building construction, aesthetics, and even unknown challenges that occur after deployment, and this is when having a consultant who simply has more experience, provides value. However, even consultants are another variable, as evidenced by many differences of opinions and designs that have been deployed all over the world. Look for systems that are deployed and functioning and apply those ideas to your needs. Next month we get back to work since Grandma and Grandpa have now discovered Netflix.

Wednesday, June 9, 2010

Chapter 6 – Free is not a Business Plan


Our system is installed and our credit card maxed out. Now, we have to either pay for it or figure out how it’s going to save what we invested in it. As an income based system, it’s pretty easy to figure out a direct correlation between expenses and revenue. If there is some kind of defined savings, we need to try and make that objective and measurable.
Let’s talk about the profit scenario. These are just the direct costs:

  1. We spent $10K putting the system in.

  2. 50Mbps costs $450 per month (data center plus roof rights)

  3. Pole rental costs $5 per month per pole (16 poles) or $80 per month
On the income side, you are going to have daily, weekly, and monthly clients. Let’s say you charge $5 per day, $15 per week, and $30 per month. It’s fairly easy to calculate your income/revenue to put a profitable scenario together. However, let’s go back to the original premise of a low cost system.
A municipal WiFi system has the basic problem of reduced range due to simply physics limitations. I plan to share additional ideas along this area in the near future but for now, let’s assume all clients are 2.4GHz and we still need 802.11G compatibility. This means that we either spend the money on an expensive, all-encompassing infrastructure, ala the sixty AP 2x2 MIMO design, or put that cost on the client side. Having the clients cover part of the Capex not only means a lower initial investment, but costs can scale upward with income.
This design took the original Muni AP concept, added 6dBi or better on the antenna gain, and had the benefit of 802.11N improvements in receiver sensitivity that adds another 10dBm. It doesn’t take advantage of 2x2 MIMO so we left 3-6dBi on the table of signal quality and bandwidth. However, we spent $10K instead of $100,000-$150,000. For 10% or less of the cost, we got 50% of 2x2 MIMO performance and 120% of the performance of legacy 802.11b/g systems. Don’t worry, there is a lot of capacity still left on the table that we can add later.
We now have to deal with the problem of not being able to connect to 60% of the indoor clients. This isn’t unique as most of the Muni-Wireless systems recommended some type of high-power indoor repeater device. Unfortunately, it was an afterthought when they determined that a high percentage of users couldn’t connect or basically that the system was grossly oversold. The indoor repeater balanced the power equation between high-power AP’s and weak laptop transmitter. The problem with these devices is that they create more interference on the channel due to that combination of high-power and omni-directional signal pattern. A better solution for the network is a directional client radio with higher gain antenna and lower power. There are many products but I suggest Ubiquiti Nanostation 2M or Nanostation 2M Loco radios. They have an optional window mount for indoor coverage and cost less than $100. They are also dual-polarity 2x2 MIMO in case the network gets updated later (hint, hint). The radios may need to be mounted outdoor for longer range or to get over the tops of houses or trees which means truck roll. These devices are not repeaters all you get is Cat-5 to the computer. Indoor wireless coverage will require a separate indoor wireless router .
How does this affect our profitability? Assuming 200 potential clients in 1 mile area, we need to get 18 clients at $30 per month to break even on the direct bandwidth costs, not including the payback on our Capex. That’s less than 10% of the potential clients in our 1 mile area, assuming all residential housing. Not an unreasonable number. There won’t be a lot of profit on residential truck rolls but at $200 per install, at least it won’t be a loss.
With 50Mbps per square mile and 70 clients, the system can be cost competitive with most wire line services. What happens however, if there isn’t a data center down the street? We have to figure out how to backhaul from a data center much farther away and probably within a LOS shot for a direct wireless. That could cost anywhere from $500 to $15,000 depending on distance, interference, and frequency availability on the roof. Although you could contact the local loop carrier and ask for a quote on bandwidth, the reality is you will pay $300-$3000 for 1.5Mbps to a 45Mbps DS-3 circuit. Some areas have MPLS and other data options but if you can get 10Mbps for less than $1000 per month from a local carrier, you are doing well.
Another option is to look for wholesale carriers for DSL. Although DSL usually ranges from 512Kbps to 7Mbps average, this goes up or down in an area based on distance to the Central Office or DSL switch. Assuming you can get 7Mbps down and 1Mbps up and your DSL wholesale carriers allows you to resell the bandwidth, you will probably spend about $60. Order 7 of them, put a Peplink 710 router on your network and you have 49Mbps down and 7Mbps up of available bandwidth. No individual gets more than 7Mbps down and 1Mbps up, but the router will load balance the users to get them the best bandwidth available. You are still below your $450 per month budget but the router will cost $4000. Peplink and other companies have smaller routers for fewer DSL lines starting at $300, so you can budget based on expected system needs. Keep in mind your oversell rate of about between 10-1 and 20-1 and that means 70-140 clients getting close to full bandwidth 100% of the time. 70 clients would generate about $2100 per month in revenue compared to your direct costs of $030 per month. The DSL idea can scale starting from 1 circuit keeping monthly costs in line with revenue.
The previous scenario is basically worst case. Assuming you have apartment complexes in the area, not only does the revenue potential increase, so does the percentage of temporary users. These are users that need 1 day, 1 week, etc… The revenue per day for 1 day users is 5 times higher than monthly users. Anything you can do to attract those users is a huge increase in revenue. Throw in areas that include business users, and the revenue potential goes up even further. Business users can be charged 40% more than residential users so there is more potential there also. Hot-Spots like restaurants, parks, etc… will add more revenue.
Here is where we are going to diverge from the original concept of mesh systems and open up the opportunity to make significantly more revenue. It’s been mentioned that the only way to really guarantee 100% performance of a mesh network is to install 60 AP’s per square mile. The reality is that it’s extremely difficult to recoup the kind of capital expenditure at $2500 to $3000 per installed AP (parts, labor, back end, and other miscellaneous costs) you need for this coverage and the monthly costs. Even our design, scaled out to its maximum potential down the road, will cost $1400 per AP installed (but it will it move some serious bandwidth). If it was easy to make a profit, companies would be throwing up municipal systems so fast; it would make your head spin. Throw in monthly costs of pole rental, backhaul or local loop costs, support, business expenses, etc…, and this model fails unless you get the following:

  1. The local government pays for use of the network thus supplementing the cost or by being the anchor tenant.

  2. Sprint, AT&T, Verizon, or some other carrier pays to hand off some of their subscriber bandwidth needs since their purse strings are slightly deeper than most of ours

  3. You find the 1% area in the country where wired carriers use their monopoly’s to make it easy to compete, there are lots of free vertical assets, and there are very few trees.
The system we designed achieves the strategy of 100% street coverage which meets most of the needs of public safety and municipalities. This opens up the government market. We have determined that some users will need indoor subscriber units. However, the one area that hasn’t been covered directly is the idea of the system simultaneously being used as a Point-To-Multipoint (PTMP) system. Basically we need a hybrid muni system. A PTMP system has the advantage of range but doesn’t provide street level coverage and usually won’t cover indoor. With an outdoor antenna on the client side, the system can support clients up to 2 miles away LOS. Our upgraded system will support up to 5 miles or more. This greatly multiplies the potential revenue of the system. Clients purchasing indoor units are creating a mini-PTMP system already. The only difference is that as the provider, you will have to provide staff that can go on-site and install a radio in a residential location. On the positive side, it can also be another source of revenue since the cost of equipment will be less than $110 for the install. Keep in mind that every subscriber we add brings in another $360 per year or more. This design with that addition, keeps the best of both worlds.
The focus of municipal networks has historically been high-density areas. The obvious advantage is having a market potential of 10,000 clients or more. These are the kind of numbers that are needed to cover a multimillion Capex. The budget model we created allows for much lower density deployment while still creating a design that creates a product that has value for public safety, water meter, parking, video surveillance, and other options that create value for a municipality to become a client. That provides two potential markets. Throw in the PTMP market, and we have not only created 3 markets, we can provide a more reliable, stable product with higher bandwidth capacity per client, and a larger coverage area.
Does this change the model of a true municipal network? Not really. Besides cellular, the most profitable wireless networks are PTMP. They cover many of the areas that wired never moved into. For example, I have an area of 50 homes that was never profitable for wired due to the length of runs. Put up a single AP with an omni-directional antenna, feed it with a T-1, charge $60 per month, and everybody wins. It beats satellite hands down and people still watch Netflix. Some part of the municipal network usually has roof rights for backhaul, usually on unlicensed frequency, to the AP’s on streetlights if they aren’t attached to fiber. Those locations are providing a PTMP system already for the AP’s. I’m suggesting that this same model be used for clients. In later articles, I will show you how this part of the network can be upgraded to easily deliver 20Mbps to residential and 50Mbps or more to businesses.
I’ve taken some heat for the fact that this system isn’t 2x2 MIMO. Keep in mind that this was first designed to create an inexpensive and/or profitable network. It will perform better than an 802.11b/g system due to better receivers, higher antenna gain, and better protocol. It can also support a PTMP design that can cover a couple extra miles around the 1 mile area for additional customers. The network is better controlled with more users using directional antennas for indoor coverage which reduces interference in improves s/n ratio. It doesn’t have 100% indoor ubiquitous coverage but it also doesn’t cost $150,000 per square mile, although it can be upgraded. We will next cover how to increase the bandwidth at each AP up to 80Mbps or so and expand the total capacity.