» Archive for January, 2005

Outsourcing: When good business ideas go bad, Part I

Monday, January 17th, 2005

I’m now well into my second experience with large-scale outsourcing. The first was in the late nineties up through Y2k. This time around, it’s with a different big IT outsourcer, and it’s definitely living up to any Low Expectations a person might have for the whole concept of outsourced IT.

First, let’s take a look at why anyone does this in the first place. Obviously, it takes two to tango–a Company and an Outsourcer. For the Company, outsourcing is viewed as a cost control measure. Pretty much throughout the period since the beginning of the PC Revolution, companies have viewed IT as a runaway cost which needed to be controlled at all costs. Thus, outsourcing is seen by CIO’s as a form of insurance against rising IT costs. The fact that it usually results in the CIO being rewarded with a nice bonus or some more options for basically admitting that he doesn’t know how to do his or her job always seems to get conveniently ignored in the deal. But I digress…

For the Outsourcer, on the other hand, the assumption is that if someone is outsourcing, it’s because they’re Really Bad at IT and/or unable to get their costs under control. Thus, there’s probably lots of fat to trim or the Company wouldn’t be outsourcing the operation in the first place. The Outsourcer then takes a guess as to how much it would cost to run the company’s IT, adds on a premium (aka “profit”) and so long as it’s less than the Company is already spending, everyone’s a winner: The Company has its costs under control and the Outsourcer has a nice, stable long-term relationship with a safe layer of profit built in. They live happily ever after.

Yeah, right.

Now that I’ve outlined why this relationship would look good On Paper, let’s take a look at the reality. First, consider the diametrically opposed goals of the two entities involved in this relationship, The Company and The Outsourcer. The Company’s goal is to receive everything they could possibly imagine for the low, low price of Free. Obviously, this isn’t what they’re going to get, but that’s what they would like to have if they were outsourcing on Fantasy Island. The Outsourcer, on the other hand, wants to be paid All The Money In The World and do nothing in return, preferably while being served fruity drinks by HervĂ© Villechaize. The reality will therefore lie somewhere between these two points.

Throw in the naturally increased “friction” that results from having all IT now go through a game of Telephone and a need to Open A Ticket for every little thing–even the off-the-record “little favors” which are the grease that get IT done in the real world–and the odds of things going well are automatically stacked against both sides in the relationship.

So what’s the worst that can happen? Actually, there are two Worst Case Scenarios. The first is that The Company negotiates poorly and signs a contract that winds up costing more money than keeping IT in-house would have. While that’s probably not a terrible thing for me, personally, it could negatively impact share price if word got out and would probably affect the long-term career prospects of the people who negotiated the agreements.

The second worst-case scenario is that the Company negotiates too well. By this, I mean that The Outsourcer agrees to a contract that doesn’t generate adequate revenue to cover their cost of supporting the contract, much less include any of that oh-so-important profit margin.

The success of the relationship long-term will thus be predicted by a combination of how far apart those two points are as well as how close to the center of that range the actual contract falls and, lastly, how close to the center each party perceives the contract to be.

If any of those factors are out of balance, then resentment will develop on one or both sides. If the contract turns out to be too heavily in the favor of the Company, then it will turn into a money-loser for the outsourcer and they will be unable to meet their Service Level Agreements (SLA’s) which can have cost impacts far in excess of any penalites assessed. If it’s too heavily in favor of the Outsourcer, I don’t know what happens–I’ve only seen contracts where the Outsourcer got so busy trying to Win The Business they forgot to consider that they would have to actually run it when they did.

In Part II, I’ll talk about the exceptions to this rule.

Outage

Monday, January 17th, 2005

As a few of you noticed, Not Bad For a Cubicle was down Sunday and most of today due to User Error. Yes, I fell victim to one of my favorite things to laugh it when it happens to someone else. Rather than shutting down the database on the server I thought I was working on, I actually shut down the database on Thurston.

And since I was busy working on the Web stats analysis project I threatened to write a few weeks ago rather than finishing up any of the articles I’ve currently got in the hopper, I didn’t notice until I was at work today and couldn’t get outside the firewall to fix the situation.

Serves me right. Anyway, it’s back now. Maybe this will get me motivated to set up some monitoring.

-your friendly neighborhood Bing

Sed quis custodiet ipsos custodes?

Wednesday, January 12th, 2005

On the surface, this isn’t all that interesting a news story. The usual, “Boy hacks major network. Boy runs amok, has access to millions of rows of sensitive data. Boy gets arrested.”

Where it gets interesting, though, is here:

The hacker knew about Secret Service subpoenas relating to government computer crime investigations, and even knew the agency was monitoring his own Microsoft ICQ chat account.

…The hacker surfed to “My T-Mobile,” and entered a username and password belonging to Peter Cavicchia, a Secret Service cyber crime agent in New York.

…The agent was also an adopter of mobile technology, and he did a lot of work through his T-Mobile Sidekick — an all-in-one cellphone, camera, digital organizer and e-mail terminal. The Sidekick uses T-Mobile servers for e-mail and file storage, and the stolen documents had all been lifted from Cavicchia’s T-Mobile account, according to the affidavit.

(emphasis mine)

So let’s think about this for a moment….I’m a secret service agent investigating an ongoing computer break-in at T-Mobile. Why the Hell would I store information about that investigation on a T-Mobile Email server? Why would I store it on ANY external server?

While I’m sure that the Secret Service has policies in place that prohibit storing sensitive information on external servers, how widespread is this problem? Not just in the Secret Service, police forces, or government in general, but across Corporate America, as well?

I’ve had to deal with C-Level executives losing BlackBerries and laptops loaded with Very Sensitive Information in the past. It wasn’t fun, but they were well aware of the potential impact of that information getting in front of the wrong people. If anything, it made my life easier long term since gave them a tangible example of the issues I deal with and the perceptions I fight to overcome day in and day out.

That was a very different struggle than the scenario this case illustrates. The definition of mobility has now shifted. Previously, managing the risks to Mobile Devices was primarily a physical security issue, with mitigation of an incident (loss or theft) being primarily handled by the use of data encryption. Devices like the Sidekick, however, add a significant new risk since information is being transparently stored or mirrored into an environment over which the user has no control.

In the article the data thief was an outsider. If it had been an insider (which is generally much more likely, anyway), odds are that it would have gone on much longer and even when the perpetrator was caught, we would never have heard about it. So how do you handle this problem? Obviously, next-generation devices are only going to become more powerful and more commonplace. As much as I as a Security Manager might like to, I can’t ban their use. Even if I did, people would still use them. Policy only works if people are on-board. If people view a policy as overly restrictive or inconvenient, they will find ways to circumvent it.

The most obvious example of this is probably password complexity requirements causing people to store their passwords on sticky notes or under their keyboard. In this case, though, I have to wonder if the agent even realized that the data was stored on a server. I wonder what percentage of people surveyed would even realize that their data was being stored on a server somewhere.

Thus, all that’s left is a combination of user awareness and vendor responsibility. Users need to be made aware up front of the risks associated with using Next Generation services & devices. No, burying the disclosure in the fineprint of their Terms of Service doesn’t count. Service Providers should be named and shamed if their privacy or security notification policies are weak. They should not be allowed to hide behind the “ongoing investigation” clause of California SB-1386.

Hopefully this particular case was simply an example of ignorance on the part of an early-adopter agent–that he believed that the information was stored on the device, not on a server. In that case, I’m willing to cut him some slack this time around. Having sensitive information stored on a device that would be a highly-tempting target to a thief is not a great decision, but the risks are acceptable, given that he would have the same risks if he carried it around in a spiral notebook. If he knew, on the other hand, that the information was being stored on a server and simply decided to accept the risk of something bad happening to it, then I would strongly suggest that he find a new line of work–his brain is simply not wired right for making risk management or security decisions.

Book-ists, Web-ists and Abhorrence of Vacuum

Tuesday, January 11th, 2005

So I’m reading this relatively-spirited debate between Clay Shirky and Louis Rosenfeld about formal versus informal naming schemes (”Controlled Vocabularies” versus “Folksonomies”) as applied to social networking and feel like I’m having a flashback to ten years ago when I was a graduate student working on my Masters of Library Science.

The faculty and students quickly split into two factions, whom I always thought of as the “Book-ists” and the “Web-ists.” The Book-ists were committed to the idea that information should be stored in Books, which in turned should be stored in libraries. If you wanted access to one of those books, you should want it badly enough that you would come to a library, search through either a card catalog or a ludicrously slow mainframe terminal (which you probably had to wait in line to use), then go find the book on the shelves, hoping all the while that no one else had been just as motivated as you to acquire this bit of knowledge and beaten you to it, either checking out or stealing the book that you wanted. If it sounds like an inferior way to go about getting access to information, it’s because it was. Thus, the Book-ists effectively removed themselves from the debate over how to apply the librarian’s view of Information Management to the Web.

Meanwhile, over in the Business School, where I found myself doing some TA’ing to make ends meet by teaching Internet technologies, including the Web, to MBA students. They could have cared less how the information was organized at a macro level. They just wanted two things:
1) To get Their Stuff on the Web
2) To make some money in the process

Meanwhile, the Web-ists back at the library school were busy arguing about what approach everyone should be taking to classifying their data. I remember debates over how best to apply the META tags in document headers, or simply insisting that the Web was too chaotic to ever replace nice, tidy information tools like Gopher. The crew over at the business school, however, not knowing what they didn’t know, were hiring programmers and graphic artists (re-christened as “Web Developers”) and cooking up the dot-com boom as fast as they could type Business Plans.

So now the World Wide Web which was coming out of nowhere with Homepages popping up all over the place and anybody could just stick a link to another page with no consideration of whether it made sense or whether that Homepage would be there tomorrow or what the heck was going on. There was no way to find a homepage except to either take a guess (IBM? Maybe they’re at “www.ibm.com”) or to ask someone you knew if they had seen a particular page.

That was the vacuum. So about that same time, a couple of guys at Stanford began obsessively bookmarking pretty much any and every site they encountered using a roughly hierarchical set of bookmarks. They then published their bookmarks on their own web server as Yet Another Hierarchical Officious Oracle. Thus was born the Search Engine, and with it the End Of Relevance for the Web-ist’s debates over how best to catalog Web sites.

Suddenly, the definition of “good” and “bad” things to put in META tags shifted away from any attempt at rational definition and was instead defined by one simple question: Will this increase or decrease my pages’ rankings in search engine results? An entire industry sprang up devoted to software and or services to help get sites indexed by search engines, hopefully in a way which improved their ranking in searches for a particular term. The Information Management experts had missed their opportunity yet again.

Now, I learn that the debate over whether only trained experts can successfully catalog data is still alive and well. Personally, I think the whole argument is moot. As the past ten years have now shown, so long as people get halfway close to right in describing their metadata, the Search Engines will do the rest. To try to argue that only experts can successfully catalog data requires you to pretend that the past ten years never existed.

Blogging and the Javascript Paradox

Tuesday, January 11th, 2005

The Javascript Paradox is something I used to joke about back when I was heavily involved in Web development. At the time, there were a tremendous number of graphic designers who’d gotten a copy of DreamWeaver or learned just enough HTML to be dangerous and thereby been able to double or triple their salaries by christening themselves “Web developers.”

I usually encountered these people because they were in the midst of learning The Hard Way that creating attractive Web sites and creating useful Web sites were very different skillsets. So while my web efforts usually looked pretty bad, they tended to provide lots of functionality; theirs, on the other hand, looked great but didn’t actually do anything.

Usually, these beautiful sites would have some bits of Javascript coolness on them that provided some bit of dynamic functionality such as graphics that changed when the mouse moved over them, creating the appearance of interactivity, and I would always ask the the Web designer if he or she had written that bit of functionality themselves. “No, I copied it from this other Web site,” was pretty much the universal response (at least until DreamWeaver began to automate the creation of things like image rollovers, at which time they seemed to go out-of-vogue. Go figure.).

As best I could tell, no one ever wrote Javascript–it all was copied and pasted from other Web sites, and this was what I termed the Javascript Paradox: If everyone copies their Javascript from other places, then where does it come from? It doesn’t spontaneously spring into being, yet I could never personally find anyone who actually wrote any non-trivial Javascript. Now I know that there are sites where Javascript developers post their work, but once again, those sites seem to mostly contain re-implementations or adaptations of previous efforts, so I’m back to where I started and the Paradox still holds true.

More and more, blogs mostly comment on and link to other blogs, and I admit that I’m about as guilty as most. In some cases, this is quite defensible, if the purpose is to conduct a written dialogue or even just to provide some running commentary about that dialogue. I think that the recent discussion between Cory Doctorow and Chris Anderson regarding market forces and Digital “Rights” Management (DRM) is an excellent example of the cross-linked discussion creating a dynamic which I think probably wouldn’t exist in another format.

Nevertheless, I realized that last week, all I was doing was adding my own snarky comments about postings on Boing-Boing, but not really making any real contribution to the discussions. This got me to thinking…are blogs really nothing more than a prose version of the Javascript Paradox? So I took a look, and what I decided is that Blogs seem to fall into one of three categories:

  1. News Aggregators — Sites that pull together news items in a related subject area. Slashdot is a great example of an almost-pure News Aggregator.
  2. Content Generators — The rarest but most also the most useful blogs. These are people Adding Value by creating thoughts where none previously existed. Pure Content Generators are extremely rare since much of the time, some News Aggregation is necessary for the new content to make sense.
  3. Me-Too Sites — These are sites that aspire to be News Aggregators, but never find any news that hasn’t already been identified by a major News Aggregator. I’m constantly amazed at (and guilty of) this tendency. These are sites that comment on other blogs, but never generate anything that someone else might want to comment on. This is my fear–that as I become more and more steeped in blogs as my initial point-of-contact for information, that I will forget that there are other ways to find things that interest me.

I mean, really, how many blog entries do I really need to see pointing to another blog entry about Time Magazine decreeing 2004 the “Year of the Blogger?” You’re now simultaneously important and invisible. This is just like my life in the Information Security business; you’ll either get used to it or get out.

Thus, I am setting a public standard for myself: If I don’t add value by posting, then I won’t. Will this hurt my posting frequency? Probably. I have a lot of ideas, but not a lot of time to type them up. That’s what RSS is for–to efficiently notify people when new content appears on infrequently-updated blogs.

When more is less

Tuesday, January 11th, 2005

The Problem of Redundancy Problem: Why More Nuclear Security Forces May Produce Less Nuclear Security (pdf) is, as the title indicates, an excellent analysis of situations where more security produces less secure results. Even more useful, the methodologies they employ to analyze the problem can be easily applied to other security problems to identify the point of diminishing returns. This paper applies to both redundancy and defense-in-depth and will hopefully prompt you to reconsider some of the current conventional wisdom regarding comprehensive safety and security architectures.

How do I count thee? Let me try the ways…

Wednesday, January 5th, 2005

Over on Boing-Boing, they’ve brought back their traffic stats and intruduced them, along with A few notes.

I was interested to see that they also use awstats as the basis for their traffic logging. While my stats aren’t nearly as impressive as theirs (No, I don’t have 200,000-300,000 readers, which is probably just as well since the bandwidth bill would bankrupt me), I do have enough traffic that my stats can be interesting.

My great frustration, though, is with the “flat” statistics that tools like awstats generate. I’ve got a number of log analysis tools all running and am finding that they produce pretty similar reports, but that none of them allow me to do the sort of reporting I’m interested in without some fairly absurd configuration and/or scripting gymnasitcs.

I’m a big fan of drill-down analysis–the ability to filter my traffic on a potentially arbitrary number of variables. For example, I might be interested in seeing if there is a correlation between browser type and traffic to a given page. This is simply not possible with “flat” tools.

I’ve done some pretty heavy-duty work on near-realtime log processing and analysis in the past (Think archiving millions of maillog entries/day or thousands of network connections/minute). Maybe It’s time to think about “scratching the itch” and writing something that does what I’m looking for.

My question to The World, then, becomes, “What sort of web traffic data analysis would you love to see but don’t?”

Cory sets DRM strawmen ablaze

Tuesday, January 4th, 2005

Over on Boing-Boing, Cory Doctorow Gets It once again. Consider this question:

If you are thinking about buying a stereo with a key feature and the choice is between two models, wouldn’t it be useful to know that in one model, the feature is guaranteed to last forever, while in the other, the feature can be revoked at any time due to factors that are beyond your control and shrouded in secrecy?

The real question, he argues, is “Not whether the device is too restrictive today, but how restrictive it might someday become.” (Emphasis original)

The DRM schemes that are being rolled out today are all thoroughly “future-proofed” by the media companies. Normally, when a design is described as “future proof,” it means that the product/technology/protocol/etc has been designed to ensure that it will continue to provide, at a minimum, its intended functionality. Personally, I find the fact that the future-proofing of DRM schemes only seems to include designing in the ability to disable features and prevent behaviors, even after-the-fact, to be quite telling. What that says to me is that the Media Companies have two interests:
1) Sucking as much money out of your wallet as possible
2) Making sure that even if someone finds an opportunity to effectively circumvent #1, the Media Company can effectively travel back in time to eliminate the circumstances and capabilities that created the opportunity in the first place.

I Hate SBCYahoo

Tuesday, January 4th, 2005

My mother’s ISP, SBCYahoo, makes no efforts whatsoever to filter spam or viruses out of their customers’ mailboxes. I also know that when I had to use their DSL for a few months last year, it was the worst Internet access I’ve ever had. Avoid SBC/Yahoo like the plague.

<plug type=”shameless”>Personally, I highly recommend Speakeasy . They provide me with both DSL & VOIP (Voice Over IP) service. I have excellent throughput (I actually get the bandwidth I’m paying for), low latency, no service “restrictions” like SBC or Verizon impose, none of the PPPoE “Dialer” crap that SBCYahoo uses as an excuse to install 350 MB of bloatware and spyware on your machine, and they have consistently have provided me with excellent support for both services without putting their call center in India.</plug>

And while I’m on the subject, PPPoE “Dialers” get an extra-dark black mark in my book. The DSL MODEM has an ethernet port. You should be able to plug in your computer to that port just like you do into your office. More importantly in the modern Intenet environment, you should be able to plug in your DSL/Cable Router (which is mostly just a firewall, network switch, and maybe print server, but that’s all that an end user typically needs or wants) and have it Just Work.

Time for a brief history lesson. PPPoE stands for “Point-to-Point Protocol over Ethernet.” PPP was originally developed to allow computers with modems attached to their serial ports (remember “COM1,” “COM2,” etc?) to talk Internet Protocol (IP, which is how pretty much every computer on the planet now talks to each other at the lower levels).
Thus, the “layers” of communication look something like this (it’s simplified, but note that the bottom of the list is the wire, with everything else “on top” of that:

HTTP (HyperText Transfer Protocol), SMTP (Simple Mail Transfer Protocol), FTP (File Transfer Protocol), etc. The Protocols (notice that’s what the “P” is in each acronym) which define how two computers ask for and receive back Web pages, send and receive email, and transfer files.
Internet Protocol This is the core of how the network hardware (routers, servers, etc) know who’s who and where a request or response needs to go. This is why there is an “IP Address,” a unique identifier, for each computer on a network.
PPP (Point-to-Point Protocol) The “language” that your computer and your ISP’s (Internet Services Provider’s) computers speak to get the higher-level Protocols back and forth across the phone line.
The Modem one on either end of the connection, they translate the computer’s requests, which are a series of electrical pulses representing ones and zeros, into audible signals that a phone line can carry.
The Phone Connection The “pipe” between the computer and the ISP. It is the high-tech equivilent of two tin cans with a very long, complex string between them.
Phone Line.

Ethernet, on the other hand, already knows how to handle everything up to the IP level. Thus, its (simplified) “Stack” should look like:

HTTP, SMTP, FTP, etc
IP
Ethernet Protocol
DSL Modems (again, think of this as two cans with a very complex string between them)
Network Cable

So why does SBCYahoo insist on creating a stack that looks like this:

HTTP, SMTP, FTP, etc.
IP
PPPoE (PPP Over Ethernet)
Ethernet Protocol
DSL Modem
Network Cable

???

From my perspective, when SBCYahoo adds PPPoE to the mix, the odds that it will Just Work head rapidly to zero. I’m a pretty hard-core techie and it took me the better part of an evening to get it all working with my DSL Router. Based on discussions I’ve had with “regular” computer users, most of them just give up before they ever get it working, which means that SBCYahoo has now effectively forced them to adopt extremely dangerous behavior so they can either save a little money on their costs maybe create a little revenue by pimping their customers to advertisers.

That’s not my idea of how you treat your customers. It’s more in keeping with the line from the old Saturday Night Live skit, “We don’t have to be nice–we’re the phone company.”

PC Security 101 for Laywers (and other non-techies)

Tuesday, January 4th, 2005

Law.com has an article titled, “Legal Technology - PC Security 101 for Lawyers.” While all of what it discusses is fairly basic and totally windows-centric, that doesn’t make its advice any less valid–if you run Linux, you’re generally not going to learn anything from this article anyway. Still, if you follow their recommendations and install a personal firewall, install anti-virus and keep it updated, and use a spyware scanner, you’ll still be a whole lot better off than you would otherwise.

I would also add using Firefox as your Web browser” and Thunderbird as your email client to my list, since so much malware (viruses, spyware, etc.) only works against Internet Explorer and/or Outlook. I helped my mother (a smart woman, but not a techie) switch over to Firefox/Thunderbird a few months ago, and after about ten minutes of Resisting Change, she was a convert.

The only warning I have about these wonderful products is that some anti-virus solutions don’t work well with Thunderbird when it comes to cleaning email viruses that make it to the mail client, because they will prevent Thunderbird from accessing its files on disk once an email virus gets into the inbox, so be sure to find and anti-virus program that can act as a “mail proxy” or “POP3 proxy.”