Post mailbox migration Exchange 2013 ActiveSync failures

There is an issue with Exchange 2013 first reported around the CU2 release date and is still an issue as of CU8.  The issue presents itself to existing users with an ActiveSynced device after mailbox migration.


  • Users experience an outage on mobile device for up to 16 hours, but typically it resolves itself within 8 hours.
  • Not all mailboxes are affected.
  • Not all devices utilizing the same mailbox are affected.  For instance, I had an iPhone and an iPad on the same account; the iPhone experienced no issues, but the iPad failed to connect.
  • Email can be sent from a device but not received.
  • Connection attempts to the server are rejected initially with a 130 – Access Denied error when viewing MDM logs or when utilizing tools such as LemonJar’s iOS Console utility.  Errors will eventually turn to IIS 403 – Forbidden errors after approximately 30 minutes of attempted connections.
  • EAS logs indicate that a user device is still being proxied to the server they were migrated from.
  • Issues appears to be completely intermittent in nature.

This is a result of the IIS application pools on the 2013 CAS not being made aware that the mailbox has moved.  The issue itself is easy enough to resolve by recycling IIS application pools: MSExchangeAutodiscoverAppPool and MSExchangeSyncAppPool on the 2013 CAS.  It may take up to 15 minutes after recycling the application pools for mail to begin to flow normally.

While researching this issue I came across an article by Jeff Guillet who has additional information and a script available on


Convergence Factors – Why I’m not sold on Hyperconvergence, yet.

Tech buzzword alert 2014, Hyperconvergence has hit the mainstream and is coming to a datacenter near you…

If you’ve spent anytime this year in the compute space or have talked to vendors about purchasing equipment then there is no doubt that you’ve been exposed to the concept of hyperconvergence.  Besides having the most corporate-appealing marketing name of all time, it does accurately describe the process.  For the uninitiated, it’s nothing more than compute, storage and network made of multiple nodes in a single box or cluster of boxes that is controlled by a single software orchestration engine.

Sound familiar?

Do you remember the days of the mainframe?  Well it’s back…sort of.  Back in the days of the mainframe (converged infrastructure), everything was centralized due to cost and complexity.  The problem was that systems were slow across large geographic distances and hardware choices were often locked to a single vendor.

Then, in accordance with Moore’s Law, systems were able to become smaller which reduced cost and the industry moved to a distributed infrastructure.  The distributed model helped make large amounts of data and compute available across large geographic areas by copying data local to sites.  It also helped make many organizations vendor agnostic.

In the last few years, with the rise of virtualization and the accessibility of high-speed networks, the industry has returned to converged architecture to help ensure data security and availability.  One of the many constant struggles as a sysadmin, is keeping all the data an organization generates safe, secure and backed up.  With distributed architecture each site needs storage.  Storage is expensive as is generating data, and therefore sites need local and offsite backup solutions to maintain data accessibility and recoverability.  This drives up costs for hardware and licensing, and if the systems are geographically distant from each other then there is a need for additional workforce or travel to manage these systems.  To help mitigate these costs and meet security compliances, virtual desktop infrastructure (VDI) has come into play and I believe is the primary reason for the rise of hyperconverged infrastructures.  VDI is very resource and network demanding and in a hyperconverged where resources are shared and load-balanced the architecture works.

Why I don’t think it ready yet?

Right now most hyperconverged product offering are vendor dependant, and I don’t like that.  Also, the architecture is so new the industry hasn’t had a chance for natural selection to kill off the failures and promote winners. The last thing I want to do to my org is spend a quarter of a million dollars on a hardware/software piece whose vendor might go bankrupt, get bought out or worse, have zero reliability.

Right now the only system that I see that has long-term promise is the VMWare vSphere/vSAN architecture but even that has some flaws to me.

What I want?

What I would like to see is a hardware and hypervisor vendor agnostic software piece similar to Docker that from the software level controls hardware and presents storage to the hypervisor to converge systems.  This means that I can choose hardware from vendors that I like to use and the hypervisor that best does the job.  At this time I don’t know of any such software.



IT, the aging workforce, and future generations.

I work for a somewhat unique company in that the average tenure is over 18 years.  This has been and continues to be incredibly valuable to my company; training costs, HR costs, and customer relationships are all better off.  It also poses a huge looming issue that is facing many companies, the retirement of the baby-boomers.  Over half of my companies entire workforce is expected to retire in the next 15 years.  This means that we will be replacing the older generation that has been in place for 30 – 40 years with middle age Generation X’s and young Millennials.

The History

Firstly, baby-boomers came with all sorts of interesting IT issues.  On one hand, they were the first generation to see computers at the college level and many of them contributed to significant advances in computing (I have the great pleasure of working with a wonderful lady who was on IBMs relational database team in the 60’s – 90’s).   On the other hand, many (most) of the generation had little exposure to computing technology until the early 1990’s.  When it was introduced into the workplace there was an emphasis on making computers seem like a mythical or magical machine, but very fragile and delicate.  To this day I have my baby-boomers asking if a 100kb Excel spreadsheet is too big to send via email or store on a server.  With baby-boomers I deal mostly with simple problems and often they can go weeks or even months with an issue that they simply just ignore or work around.  They love to print email, spreadsheets, anything.  They understand an honest days labor, showing up at the office 15 minutes early and leaving 15 minutes late.  I’m going to miss them.

Gen X’ers, this to me is where the problems are; they know enough to be dangerous.  The whole time they were in college the emphasis was that computers were the future.  Automation systems were completely taking over factories and the youngest of the generation grew up with computers in the elementary school classroom.  These are the “hip” 40 somethings with an iPhone, iPad and MacBook Pro and can’t use them to save their lives.  They remember the days of dial up internet and slow/no networks.  They understand files take up space and that it’s a bad thing (delete ALL the things).  They excel at generating revenue using computer technology unless they are distracted by trying to understand why Facebook is updating again “for like the third time this month”.  They want (and often fail) to find better ways to complete task and will waste endless hours trying.  They view the office as a social hub, and show up and leave the office precisely on time.

Millennials, our future.  Never a day without a computer in the home, and had parents that were distracted by laptops and cellphones.  They view computing in a completely different way.  They don’t concern themselves with network speeds (unless it’s slow) or file sizes, it’s all about data availability.  These kids grew up with broadband in the home, bandwidth means nothing to them.  Literally, they don’t understand it; the same goes for data.  I was speaking to our photographer and he couldn’t understand why we could just “give” him 4 TB of production storage for his RAW images.  He had no idea of the cost or work required in doing so.  Millennials don’t see the workplace as a static place, it’s more of philosophical state of mind. To them, work is time spent on a email at a cafe, at home watching Game of Thrones while connected to the VPN, and taking a call when in the car on the way to a concert.  They want to work how and on what they want to work on.  They won’t even consider working for you unless their technological needs are met.  They are driving for the Internet of Things (IoT).  The office doesn’t need to exist in their minds, it’s an archaic prison of the past.

What does this mean for IT?

Goodbye handholding baby-boomers, hello whining Millennials.  Be prepared to spend more money on bandwidth, multiple ISPs (heaven forbid the internet (Facebook) go down for even 15 seconds), diskspace and the almighty Cloud.  Bandwidth and diskspace are not the rare commodities they once were.  Network and data security are going to continue to be areas of concentrated focus as users move out of the traditional office to anywhere there is an internet connection.  Be prepared for a huge influx of network (specifically WiFi) connected devices as the IoT takes over the straggling remains of the office space.  To survive, you’ll need to understand multiple ways of delivering applications to the new fluid workforce.  As a matter of fact, HR is going to demand this since they won’t be able to retain the workforce without it.  Rather than rolling out standardized devices for everyone in the workplace bring your device (BYOD) is going to be king.  The distributed network infrastructure to branch offices is evaporating into the hyper-converged networks that offer nearly 100% uptime to meet growing demand from Millennials.  As all this happens, sysadmins will need to move away from managing individual servers and towards managing automated application delivery platforms.


Computer science, Information technology and why I probably can’t help you.

First, I want to be perfectly clear that this is not a diatribe directed towards anyone, or even a diatribe at all. It is an explanation, with the hope that it will help more “users” (for the lack of a better term) understand why “computer” people (again, for the lack of a better term) can get irritated by questions related to their field of work. Not all technology jobs are the same. You wouldn’t say that a miner, a steelworker, a millwright, an ironworker and a jeweler all possessed the same skill, and yet they all work with metal, right? For many of us in the tech sector, we are often expected not only to know why the game on your phone is crashing but also be proficient at playing it as well (cause your a nerd, right). Just like a steelworker and a jeweler both understand the processes of melting and moulding metals in to useable objects, there is overlap in knowledge in IT, but it is NOT the same job and probably shouldn’t even be considered the same field in many cases. As information technology is rapidly expanding into every part of our lives, the knowledge of that expanse can in no way be covered by a single individual. I work as an IT generalist (my job title is system engineer), meaning that in a given day, I might be working on a mobile phone, a server, a storage device or any one a myriad of network devices. This means that I probably don’t know what TV you should buy, what PC is best for your kid’s new game, or what boolean operator should be employed on a college programming project you are working on that’s not producing the anticipated results. That’s not to say that I couldn’t figure it out, that’s part of what I’m good at, but I probably won’t have the instant, “canned” answer you are looking for. The next time you meet someone that tells you they work with computers, please don’t immediately jump into the problem with your 401k’s website and what would I suggest they do to fix it. Instead ask, “What do you actually do in the IT field?” There are some really interesting, cool jobs out there and it’s someone’s job to do them. Take a few minutes and learn about one of the many parts of this brave new world.