Recently a curious soul put out an open question to the general LinkedIn test manager community at large about whether they, as test managers, or some one else writes job descriptions for openings about to be posted publicly or internally. As with most internet questions posted under the false guise of anonymity, there was more than likely an ulterior motive. Since we can assume there is one, let’s just move on as its probably something as banal as they’re miffed about a job they accepted having little to nothing to do with the posted description to which they applied. I can answer that concern with a simple statement, no one does the job listed on the description for which they applied. Even the person hired as a ditch digger will more than likely be someday soon asked to dig a trench, a drain, or a trough. The medieval poetry master’s candidate then has three choices: they can shrug their shoulders and think about rondelles while they dig; they can walk off the job because they’re a union certified ditch digger and if the company wanted a trench they should have hired some one from the local 147 branch of International Trench Excavators; or they can dig the hole and complain loudly to anyone with no actual authority to fire them about the injustice of it all.
Guess which one the people complaining about job descriptions usually fall into?
Of course there could be an actual issue here, which is a failure to match up stated expectations with the actual work some one performs on a daily basis. The issue of work mismatch is a lot more common than you would think. You can easily check on whether you suffer from it by doing a simple test. Do you know if you’re doing a good job or meeting the expectations of your boss without needing to go ask them? If you didn’t need to think about the answer to this question, you’re either a cocky bastard or you are working in a position with clearly defined expectations and success criteria relevant to your daily work. In either case you’re lucky because either you can spend your time focusing on actually doing work with a clear idea of how to advance (if you should want) or you’ll probably do well in life anyway if you can back it up because cocky bastards usually do if they can.
For the rest of us, a lack of understanding usually indicates we’re working at a position where our expectations and success criteria are either ill defined, subjective, conflicting, or completely at odds with our actual work. There are several reasons for this but the most common I’ve seen is revulsion at the thought of doing “busy work.” Most people at most companies consider things like role definitions and such to be odious tasks best left to others with no actual work to perform. Real workers are seen as too valuable to waste their time on bureaucratic nonsense that will soon be outdated soon in any matter. The trouble is this hurts companies in two ways. First it hurts morale because workers don’t have a clear idea of what they are expected to be doing, how to judge their own performance, and what steps they need to take in order to get that raise, promotion, or bonus. Second it hurts recruitment because each job opening is then treated as a special snowflake to be treated as an individual miracle or they’re treated as identical cookies cut from the same form with a few differences in toppings for decoration.
As an example, without a clearly defined set of expectations for a role the hiring manager, recruiter, and agent are all forced to guess what is now the difference between a senior development engineer and just a regular development engineer. Taking it even further, perhaps there are no senior development positions even though your needs cover a wide range of experiences and skill sets.
I know your company is different. You’re showing the world what a ragtag group of misfits can accomplish without all those titles and rules, man. Our founders call themselves “Head Necromonger” and “Chief Gelfling” respectively. I would then submit that come time to hand out bonuses or stock options or whatever you want to call it, there are going to be people who wonder why one gelfling received a larger share of stock than the gelfling working across the cubicle wall. The recruiter you hire will also undoubtedly wonder what the hell a “necromonger” is and which Monster group should they post the opening into. Is it computer software or hardware or is it a religious thing and should therefore go into nonprofits?
If you spend the time to accurately document the roles and their expectations within your organization, you will find your workload as a manager eased more than you can now comprehend. Performance reviews then become a fun activity where you can work positively with some one to advance their careers or shore up weaknesses preventing them from getting the recognition. Conflict management within the team becomes so much easier since most of these issues fall into the “not my problem” category. But most of all, when you finally get the okay to hire that new team member you’ve been following the director into the bathroom to lobby for … you’ll actually have most of a really good job description completed. Just grab the role description and add the fun bits about specific skills or needs and run it through HR to make sure nothing you don’t want public will get out.
And then its done and you can then turn your attention to sorting through the 500 resumes on your desk because it turns out there really is a job called “necromonger” in Utah and the primary employer for that industry just announced a staff reduction of 30%.
Good luck, my little gelflings.
Within the theory of general relativity there is a theoretical entity called a “white hole.” A white hole is differentiated from a black hole in that nothing can enter from the outside but particles of light and matter can escape. In this respect it is the opposite of a black hole where particles of light and matter may enter from the outside, but nothing can ever escape. A white hole is a useful reference when discussing seemingly insurmountable problems simply because people often forget about them, preferring instead to simply reference black holes for everything.
As a case in point, I recently(ish) heard some one advise a hopeful candidate to not bother with throwing their resume “into the black hole” that is the primary corporate recruiting website of a large software corporation. Granted they may have been actually referring to a hole in the ground of indeterminate depth, but that doesn’t bear at all on my point so I will simply ignore that and other possible interpretations. See how it works when you’re a blogger? In this world I’m all sorts of witty, mainly because I get to ignore that which makes me seem loutish.
In my witty world where I also have a flying pony named Sir Muffintoes which I use for commuting to and from work, it occurred to me the corporate career website was not a black hole out of which everything enters the corporation and nothing exits. Rather it is the reverse. A wealth of information about a company is regularly spewed out of the corporate career website on a regular basis. You can find all sorts of fun information if you know where and how to look. For instance, if you see a particular team has a lot of open requests for new hires you can safely assume either this team is expanding their charter scope/vision or they recently laid off a lot of people and are looking to replace them quickly. You now have some “insider information” to use when scanning industry trade publications, discussion groups, and other online information sources. If you target that group, at worst you will discover what’s called a “disgruntled ex employee” who will probably be more than happy to trade all the bad information they’ve been storing up for a few kind words and possibly a latte. At best you will find out the group has been experiencing rapid growth or a sudden injection of capital and exposure from the corporation, meaning it will more than likely be a little chaotic with new spaces you can stake out as your domain without fear of getting into a turf war with an already entrenched five year veteran. In short, any information you can pull from sources other than press releases, corporate communications, and paid for reviews is endlessly useful … and corporate career websites are chock full of information.
The one thing they are not good at (in well known and/or large companies) is getting you noticed by anyone in human resources or recruiting. I’ve never seen it personally, but I have it on good authority from a person I’ll call “Rose O’Sharon” the company she (or he, because there’s nothing wrong with that … really) works for receives an average of 1,000 resumes a day submitted through their corporate career website. Granted that’s for several hundred open positions but that’s still an awfully large herd in which to have your horns gleam in the sun. Rose also informed me because of this bull load, almost every resume submitted is sent straight to an automated parsing engine which sucks all information it things is pertinent out of your resume and stores it in a massive database. The recruiters and HR folk all subscribe to a feed which sends them little love notes if the system detects a bull with Java coding skills, SOAP experience, or possibly black spots. Rose then calls up the entry in their system and checks to make sure it really is Java coding experience and not work history at the Java Bean during college. Rose then scans the entry and if s/he likes what s/he sees, then s/he will pull a copy of the actual resume to review. If the system can get the number down to 10 a day, Rose is happy.
Now I notice from my vantage on the back of Sir Muffintoes you are holding up a finger and proclaiming, “A ha! Rose said ‘almost all’ the resumes were sent to the system. What happens to the ones that aren’t?”
Good catch, my little rapscallions. Some of the resumes are refused out of hand for reasons ranging from language usage to something called “black balling” where a person is barred from applying to a company for a period of several months to forever. But there are a magical few, rare in nature, who are submitted directly to a human. The problem you face is you will not know which of these positions do that when you apply, so its like purchasing a lottery ticket in the hopes of paying off the loan shark. Sure, it may happen. Just don’t be on my couch later when there’s a loud knock on the door.
There is a proper way to submit yourself through the corporate website, however. The correct manner is after you’ve been requested to do so by the recruiter or hiring manager directly. Oftentimes they’ll even go so far as to ask you for the confirmation number so they can pull your account up. Why do they do this? Because the system has more of a tendency to miss sending them good candidates than it does sending them bad ones. They’re designed to weed out resumes and not find good candidates, so in this respect they succeed admirably these days. Of course even if you are a perfect candidate, you have about a 98 in 100 chance of being ignored by the system. So the recruiter/HR representative asks you to apply and send them the confirmation number.
So how do you get to the recruiter or HR rep in the first place if you don’t go through the website?
That’s the next entry in this series. The next one’s all about how there is no shame in asking everyone you know if they can help you, even if you haven’t talked to them in years or may have possibly given them a rash the last time you met.
I’m thinking of calling it, “Employee referral bonuses can erase a lot of resentment.” It’s a working title.
A common question among project management staff is who should make the call to release a product, when they should make it, and on what basis they should judge. I can now definitively state I have uncovered the answer. I’ve been reading through historical records of past projects and have uncovered a system whereby countless projects from other disciplines were successfully launched to great acclaim. The system is ingenious in its simplicity in that all you need is a flock of birds and some one you can pay to tell you what it means when they fly in certain patterns. The Romans were adept enough at this practice to have kept detailed documentation on patterns, weather, calendar dates, and breeds and how each factor was interrelated into a complex system. They called these books “historical data” which they used to analyze their “metrics” before giving the blessing of the gods or not. The beauty of this system was twofold, first it guaranteed an income for the second and third sons of aristocartic families who otherwise might be forced to become farmers or merchants or something actually productive. Second it completely removed any responsibility for the eventual decision from anyone who might be punished for it or otherwise blamed later.
I apologize if this sounds like the rambling digressions of a bitter and possibly demented turnip, but the practice of augury and their practitioners (the augurs) have been much on my mind of late. I’ve also been thinking a lot about the ordeals of Hercules, particularly his task to clean the Stygian stables of manure. It was a seemingly hopeless task as the stables were so large that any effort to clean one area would mean the others ended up being fouled in the meantime. I know its not technical jargon, but we can’t all be computer science graduates now can we? Some of us actually wanted chat up girls in college, so we followed them into the Arts department.
My Stygian task of late has been to address some modern day augurs who seem to believe you can predict the point when software is to be released by studying the flight patterns and behaviors or software’s version of birds, or rather tests and defects. Every time I think I’ve addressed the subject with some wit and wisdom, I find the blessed group has snuck back into the stables with their wheelbarrows full of charts and studies ready to spread around with their loamy gooodness.
The ugly little truth about releasing software to the client is there simply is no objective metric by which you can judge. The emperor has no clothes. The birds fly that way because some one spilled grain on the street just over the wall. Or worse yet, they’ve been trained to fly that way because the augurs know you’ve done this enough times to have a few expectations. The fact of the matter is releasing software is about intuition. Informed intuition, but its still intuition. Some one or some group has to be tasked with “making the call” and they have to be given whatever information they need to make said call. But at some point, the call has to be made because not making a call is in and of itself making the call not to release. So-called objective metrics may be informative in the macroscopic realm of process improvement when analyzing multiple releases across multiple cycles and products, but they are somewhat useless predictors for specific instances. They can be among the information requested by the decision maker(s), but they should and cannot be considered the decision makers themselves.
Perhaps it would be better to switch metaphors and start talking about cricket or baseball. Both games appear to be ruled by statistics. Batters and pitchers/bowlers are swapped out because of a perceived advantage against a particular opponent based on statistical performance against similar persons. In both games the person batting must commit themselves to an action before the moment for that action (ie: the ball arriving) occurs due to the high rate of velocity at which the action is occurring. Both decisions, whether to swing and whom to play, are informed intuitive decisions based on statistical analysis and talent, but the analysis and talent cannot ensure the decision is correct. That’s why the games are (sort of) exciting (to some). Statistics can tell the batter the picther/bowler is likely to throw the ball a certain way, but they cannot assure him or her of that throw. The batter must commit to swinging before they know the actual outcome of the pitch/bowl, either connecting or not.
The same is true of software releases. Talent, experience, and statistics can help inform the decision that must be made, and any decision maker would be criminally foolish not to avail themselves of said resources. But passing the cup of responsibility onto the statistics themselves is similar to swinging your bat or not because statistics state the pitcher will now throw a curve ball into the lower left pocket of the batter’s box. Or to put it more appropriately, it would be like declaring war on the Etruscans because there were three white doves in the flock that flew south while thrumming. The war or the project will continue under its own momentum, but the decision makers charged with declaring war or release can absolve themselves of blame while reserving the option of showering themselves with glory. In reality the only ones who win are the augurs who charge for their services.
The rest of us just have to go about our business and clean up the droppings.
I’m going to continue on with my advice series on pursuing a career within the warm and nurturing bosom of a large (5,000+ employee) company. In the spirit of complete and brutal honesty (to be elaborated on later) I am forced to admit this series is becoming something of a chore for me. This particular subject is more of a chore than all the others combined. Why is it? It’s probably the most useful advice in the entire series yet I suspect it will also be the most often ignored, intentionally misinterpreted, or outright dismissed. It will likely be dismissed because people will just assume the advice does not apply to them or to their situation.
Why do I bring this up? Because it’s part of the next stage in your career search, and its one you should never stop practicing as my experience recently proved to me. The next step is to take the corporate information you’ve gathered about your company, division, department, and personnel and honestly weigh yourself against what you’ve found. Coldly efficient honesty is the key to this exercise. You cannot, and I repeat this emphatically, you cannot rationalize or deal your way into a job with a large company. The process is simply too long and has too many layers designed to weed out just such behavior. Sooner or later some one will hear you typing their question into a Google search or will ask you to walk up to a blank white board with a dry erase marker in your hand. Tricks like repeating the question just asked as you stall for time or throwing the question back at the interviewer will not work as sooner or later you will face some one who knows what your’e doing.
So if you’re going to be coldly efficient in your honesty and self-reflection, on what areas should you focus that attention like a laser. That’s easy. You need to be realistic about the following things:
The first two questions are complementary so I’ll address them together. The first thing you should know is job descriptions always lie by omission if not outright false statements. It cannot be helped. The job descriptions in large companies are usually generic documents that if anything are simply edited by the hiring manager should they have the time, inclination, and perceived need to take on the HR department. You should have already peeled back the onion to find out more about the people actually hired by your dream team. Now is the time to assess them for common technical and non-technical traits. A common issue is whether or not some one seeking a product team position (ie: tester, system engineer, product manager, director, etc) will be expected to code. The answer is almost always “yes” by the way, even for management or product research positions. But you should be able to answer that question definitively for the specific company. You should also be able to answer questions about their preferred technologies, tools, languages, and methodologies. Non-technical competencies could include financial markets information, drug trial protocols, regulatory compliance standards, or literally anything not related to software but which you will be expected to know as part of your job. And before you ask yes, I’m sure some of these companies will happily spend the money and time training you on these non-technical subjects. However if we assume a company like Google will receive an estimated 500 resumes in a single day, why should they? Sooner or later some one with that knowledge will present themselves and you’re expenses won’t be necessary.
Non-technical requirements leads us nicely into a fuzzy area, personality. While its true large companies tend to have a certain homogeneous feeling about them, they all have an iconic personality to whom they still associate even if the reality is far from that. Apple employees still like to think of themselves as creative revolutionaries seeking to bring down entrenched megalithic corporations with their visionary ideas and products. The reality is they are one of the largest megalithic corporations in the world and will sue some one and their family into debt slavery if they feel their intellectual property has been sniffed around, much less breached. But who do you think they want to hire, the vindictive little climber who will scratch and claw their way through the dirt to get ahead or the idealistic dreamer willing to sacrifice everything from their health to their family for that dream? I apologize if it looks like I’m picking on Apple (please don’t sue or make my wife switch to a PC, she just finally stopped asking me support questions); they’re all like this. The iconic Google employee is a counter-cultural individualist who supports open source software and the free exchange of ideas. The iconic Microsoft employee possesses a genius level intellect which they use to help save the planet while biking into work every day and recycling their plastic bags. It doesn’t matter what the reality of their day to day business actually entails, this is how a lot of them see themselves. You need to ask if that’s how you see yourself as well, and if you can handle not being that anymore.
Now comes what I call the hornets’ nest question. Everyone claims they are committed to diversity and want to hire the best and brightest minds from around the world. Don’t believe it. You’ve already researched the company/department and its employees. Ask yourself honestly if you are like them. If you’re applying for an internship at a major financial services company and all the current employees are graduates of the Ivy League with last names like Stevens, Pendarvis, Johnson, and Aldrich, you’ll need to seriously ask yourself if some one with the last name Parmahattmani will have a serious shot at a post. This may not be fair and it may not be popular, but its honest and its pervasive in the business world for a simple reason outside racism, xenophobia, or sexism. People like to hire people they feel comfortable spending time around. A typical job can expect to claim one a little less than one third of your life while you’re there, which means a hiring manager will put a lot of time finding some one who “works well with the team.” This can result in some really silly if not unethical situations. A pretty woman might be passed over because a manager will fear his team of socially undeveloped nerds might feel intimidated around her and slow down their performance. An immigrant programmer with a doctorate in engineering and several well received articles might be passed over because the some one in the loop complained they couldn’t understand a thing they said because of a slight accent. An experienced manager will let slip he and his wife are expecting a child which will take him out of the running as the hiring team will simply assume he will soon be asking for paternity leave, taking sick days to care for runny noses, and not available nights and weekends to meet deadlines. So it’s time to be perfectly honest with yourself to avoid time spent fruitlessly. Look through your research and ask yourself if those people are ones you could see hanging out with socially, or wanting to do the same with you. Do you share similar interests, hobbies, and backgrounds? If not, you’re welcome to still try and you might get lucky. However my advice is to try someplace that will actually value your contributions. Don’t let it get to you as there are literally hundreds of companies out there looking for talented people. Any place that doesn’t actively practice diversity is only ensuring their own demise. Besides, companies stocked with a healthy diversity of backgrounds, cultures, and viewpoints are ALWAYS more fun to be at. Always. And not just for pot lucks.
Finally there’s the single most difficult question to answer truthfully. What sets you apart? As we stated before, a large company will see hundreds of resumes submitted in a single day from qualified and skilled candidates from around the world. Part of the allure of large companies is they attract great candidates like ants to spilled sugar. Part of the problem is making sure you are noticed as the colony converges on the mound. Make an assessment of the skills you think would appeal to the company or team, then ask some friends to do the same if possible. Anywhere you see duplication, cross it out. What’s left is a good indicator of what you have to work with. If you’re left with nothing, go get something. Perhaps you know this company values certifications you could earn, as is common in the healthcare software or device industry. Perhaps they value people well versed in their products already, as is common in the video game industry. Whatever it is, highlight those attributes. The trick is to make sure its pertinent to the company and role to which you’re applying. If you’re applying for a technical writer position it’s probably a good thing to be already published within the industry, but no one’s probably going to care that your fan fiction blog has over 2k unique views a month. They won’t care, unless you’re applying to Lucas Films and your running a popular Star Wars blog. So all specific advice is as they say, useless. This is work you need to do on your own and do it honestly. If you don’t have anything to offer above and beyond the norm, you probably won’t be given the chance.
Modern (post-industrial age) sociology has created a few useful fictions or stereotypes which have found their way into the general parlance. True, most times the general understanding of these fictions is completely inappropriate or outright incorrect when compared against the actual usage meant when coined. But one thing is certain when it comes to concepts being taken out of context, no one really cares except the people who originally formed them.
I bring this up because one of those useful fictions is causing me some issues lately, and I’d like to address it. The fiction is the supposed tension between conservative and liberal elements within a society. Before knickers are bunched and assumptions are brandished like weapons, let’s address the terms themselves. Conservative elements those elements resistant to change while liberal elements actively promote change. Supposedly there is tension within both micro and macro societies between elements attempting to defend the status quo and those seeking to alter it, the ascribed motivations for each usually being a function of one’s particular belief system when it comes to sociological or political or economic theory.
The issues I’ve been having lately are warring camps who have decided to stake claims within the quality assurance community around sequential versus iterative development methodologies. Sides have apparently been chosen and mental territory is being defended with the vehemence of children on a playground. Accusations ranging from luddism to anarchism being tossed back and forth with wild abandon. Of course like most dogmatic debates based on theoretical constructs, these completely miss the point.
So what is the point, you may ask? The point is appropriateness. Clayton M Christesen’s book The Innovator’s Dilemna (required reading in my opinion) raises the very interesting point that established companies are successful because they have developed processes which drive or facilitate their success. In the case of innovation in alignment with these processes, he deomonstrates established companies have a remarkable track record for staying ahead of the curve and leading their market in both quality and innovation. It’s only when the market changes due to what he terms “disruptive innovations” (ie: 3.5 inch disc drives, hydraulic excavation tools, dirt bikes, etc) that established companies fail while smaller companies succeed.
Mr Christensen’s (paraphrased) explanation for this is simple, the processes in place at successful companies cannot be drastically altered in the face of disruptions. Successful companies can bend and adapt processes, but only up to a point. What do I and he mean by that? Let’s take a look at solid state disc drives. The movement from 8 inch to 5 inch disc drives was disruptive because it reflected a radical change in the user market with different needs and requirements (personl workstation vs PC). Does the shift from magnetic to solid state drives represent a similar shift? No. The solid state drives currently being marketed are used in the same manner as a magnetic drive, by the same customers, and represent a significant leap in performance (but not stability). A more interesting disruptive technology is the use of SD cards, which with the advent of SDXC have the possibility of storing as much data as a normal hard drive with the added benefit of an incredibly compact size and amazing portability.
The follow up point Mr Christensen makes about disruptive technologies is the creators nearly always get it wrong at first when predicting markets. Xerox got it wrong with the mouse, and so gave it to Macintosh. Honda got it wrong with their small 150cc motorcycles, but got it right later when outdoor enthusiasts in California began buying them for off road fun. The lesson to be taken from these examples is a company working in a disruptive technology must practice learning development and not commit their resources to a particular path. They must work with the markets, experimenting with offerings, and altering their designs and assumptions at the requests of the market rather than assumptions or market research projects. If a car company wants to purchase your 3.5 inch disc drive for a navigational system and cares more about durability than write times, start making them more durable and stop working at increasing capacity and write times Does that sound familiar to any of you? It should because that learning as you go model is capital “A” Agile defined for all industries.
So what does a quality assurance professional do when there are no defined processes to review as the company is learning what those should be? Plenty. The quality assurance professional in a disruptive company should act as the opposite of a process improvement expert. Companies looking to build out a disruptive market cannot afford to define their best practices until they have established themselves as successful. To do so is to miss market opportunities, allowing another company to grab them. So the role of a disruptive QA is to constantly act as an iconoclast while making sure all market opportunities are explored, even if they at first do not seem to be in alignment.
So to summarize, there is no one way to do things for all companies. Processes like CMMI ad Six Sigma are highly effective in established companies seeking to lead through sustaining innovation (ex: solid state drives for the PC). These same processes are death to a company seeking to lead through disruptive innovation (ex: SD cards with embedded operating systems) as they will inevitably result in missed opportunities and inflexibility. The role of QA in sustaining innovation is to constantly work at improving processes by looking for inefficiencies, waste, duplication, or other sources of “muda.” The role of QA in disruptive innovation is to constantly work at making sure processes are supporting the learning development rather than defining it. In a disruptive company there is no “muda” as there is no frame of reference for waste.
I know it’s been a while since I touched on my much vaunted and read (but not commented on, you sneaky boots) series imparting advice for people wanting to attract the attention of and then start working for a large software development company. It’s been a while, but it hasn’t left my thoughts except when something more important takes it place. Something like ice cream, ponies, parties, or parties with ice cream and pony rides. I know. I’ve been going to my happy place a lot.
In case you missed the earlier posts, I debunked the myth employment at a large corporation is any more stable than at a small startup company. I next talked about bureaucracy and politics and their ubiquitous presence in all businesses, but to a greater and more personally frustrating level at large corporations.
So you’ve read those two posts and still think life in a large corporate machine sounds pretty good. Let’s not kid ourselves, it can be a blast. Small companies can’t usually afford to spend years on a research project that may or may not result in a directly profitable line. Small companies don’t usually have the budget for sending employees off to the far corners of the world to interact with leaders in the industry. Heck, small companies can’t usually afford to help relocate a rock star employee who would be perfect in every other way. So you’ve decided you want to get a job with MegaWidget and see where the path takes you.
Great and good for you. The very next question you should ask yourself is how much do you really know about MegaWidget as a company? Chances are very good you don’t know as much as you think unless you just happen to live near the primary corporate headquarters and are golfing buddies with all the senior management who are liable to start spilling company secrets over Manhattans in the lounge. Of course if you are one of those people, please contact me immediately as I would like to add you to my contacts list on LinkedIn. I should be asking YOU for advice on landing a job there.
For the rest of us, we need to answer the following three questions about a company when looking at applying.
Corporations are not perfectly aligned bodies sharing a single vision and culture, no matter what the press releases and corporate recruiters may claim. Different groups will be made up of very different people with different processes, values, and culture in spite of the best efforts expended by corporate human resources and process control. For instance, the culture of a group tasked with developing three dimensional mapping software for military guidance systems will be about as radically different as you can imagine to another division developing online casual games. And yet if you were to cruise the “careers” section of the corporate webpage, I would venture to guess both teams would be advertising for test engineers with experience in three dimensional mapping technologies. Depending on who actually wrote the position opening, they may even use the exact same phrases and words. It’s up to you to dig deeper into company publications, interviews, and online chatter in order to peel back the layers of the groups. One of he worst things you can do is simply send off your resume willy-nilly for everything because large corporations keep track of that sort of thing.
Once you’ve figured out which group you’d want to be working with, you’ll need to ask yourself how they’re actually doing in the marketplace. Are they doing well, or are they being kept alive by artificial means? Being stuck on a project that’s sucking funding from other groups due to profitability issues is actually less fun than working on a small startup in danger of going under. At least on a small startup, there’s market pressure to either put up or shut up by knuckling down to fix the problem or just closing up shop once and for all. Once the “all in” decision has been made, it can be one of the most exciting jobs you’ll ever have as I’ve been told its like taking a ship through a storm. Everyone pulls together and either comes home safely or goes down together. On a product simply held together by corporate life support, you often find only inertia and the feeling your co-workers are all there with you because no one else wants them and no one can be bothered to fire them. Asking yourself how a product is performing will help you figure out what you’re walking into.
Finally you’ll need to take a good hard look at the group’s performance over time. It doesn’t matter if it’s a “new team.” In a large corporation there is no such thing as a new vice president, director, or manager. They all came from somewhere else, which means they have a history you can research. Does the team itself have a good track record of responding to new challenges and disruptive innovations? If not the team, does the management? Has the team consistently “knocked it out of the park” with their offerings or did they just get lucky once and coast on that win ever since? How likely is this team to not only survive, but evolve and grow as the markets change? These are all vitally important questions as you’ll need to be working with this team for a while, even if your plan is to request a transfer to the Rio division just in time for Canrivale. New employees don’t get plum assignments or transfers. Those plums are reserved for the people who’ve built up good capital with management over time, which is another reason to question performance over time. It’s impossible to build up that goodwill and capital if management is constantly being replaced in order to “respond to the new challenges.” A team that’s flexible enough to respond or deeply entrenched in an industry adverse to change or challenges is the best way to quickly build a good name for yourself.
Now I know what you’re thinking to yourself. You’re thinking, “I don’t live in Metropolis and have friends working at MegaWidget I can ask about the real workings of a company. How can I possibly find this stuff out?” Easy. Don’t read the press releases or the corporate website literature. Hit the books, armed with your questions. Start trolling the trade blogs and messageboards. Follow the company on sites such as LinkedIn and BusinessWeek. Start reading the actual trade journals and leave off ones like PC Mag and CNet. Dig into the biographies of the upper management and start asking around about them. Before you know it, you’ll soon know more than 80% of the “journalists” out there who seem nothing more than tools for editing down press releases from 800 to 250 words or less. You’ll soon have an idea if you really want to go work for MegaWidget or one of its competitors you identified. You’ll also know to which groups you’d stand the best chance of appealing both technically and socially. You’ll also have started work on your indispensable list of names you can either contact or reach out to indirectly with references.
All this information will become vitally important as you move through the next phases of this process, taking honest stock of yourself and knowing who to contact and how to contact them when starting the process.
I’ve been getting an education lately in something called “test oracles” by a couple of people I know and have come to respect as far as those things go. Initially I was surprised I wasn’t aware of the term before now, but then I wasn’t. Chances are I’d heard it and then promptly forgotten it. I don’t think I’ll be able to clear out the clutter this time, so I might as well defend my decision to shift-delete the thing if I can.
The context of this re-education came about during a discussion about negative testing which, as usual when I’m involved, quickly branched out faster than the Mississippi delta into a wide range of topics from the appropriateness of negative testing to the use of testing itself. The subject of test oracles came up when I posited the role of testing was to check and validate assumptions about the functioning of a product. This caused quite a stir, probably because I used the words “assumption” and “validate” within the same sentence. I defended my stance by making the claim we could not know if a test was “passing” or “failing” if we did not have some expectation of assumption of how we expected the system to function. If the point of a test is to actually learn what something does, then any behavior up to and including smoke and fire leaping out of the housing are not failures. Of course neither are they passes. Experienced behaviors in a learning system are all simply experienced. They live in the weird transitional state until some one makes a determination, labeling them as either a pass or fail. This is or course what I meant to say. But since the conversation was proscribed into 120 characters, it lost something in the translation.
I was then told/scolded by several people that we most certainly do not use assumptions when conducting testing. We use something called oracles, and only oracles.
My initial reaction to this was something akin to shock. My education and interest in classical Greek mythology as well as world religion and anthropology made me perk up my ears at this use of a very specific term. You see oracles were (and are) very specific roles within societies spanning the globe. Oracles received instructions from divine sources directly and were thought to relay them directly to humans. People within a society visited an oracle when they wanted to ask a question of the divine directly, with no mucking about using entrails, star positions, runes, or other nonsense. Visiting an oracle was something not to be taken lightly, by any means. It was generally thought the divine didn’t appreciate being disturbed with questions about a lost ring or your neighbor’s donkey getting into your garden. Some cultures even believed consulting an oracle would then bring you to the personal attention of the divine, which was not generally considered to be a good thing. Finally the pronouncements of the oracles were often mysterious and prone to misinterpretation. Tragedy in Greek literature, for instance, is primarily based on humans attempting to interpret oracular pronouncements and either seeking to avoid their supposed fate or just outright getting it wrong. A good example of this would be the father of Oedipus, Laius. The oracle warned him if he fathered a son, his son would murder him later in life. Laius then set to kill his newborn son by leaving him out in the open, exposed to the elements. Long story short, Oedipus survived and later killed his own father not knowing it was his biological father. The tragic element of this story was had Laius kept the baby in his house, he would have known his father and probably not killed him In other words, the statements of the oracle while true were not useful, informative, or even enlightening in any way. In fact they were the opposite as if the king had never received word from them, he would probably have simply raised Oedipus as his son and heir and gone on to live a long(ish) and happy life.
Are the statements of these people really what you want to base the determination of your testing on? Really?
Then I began to dig a little deeper and realized the term does not really imply an oracle in the classic sense. A test oracle is “a mechanism for determining whether the program has passed or failed a test” according to Cem Kaner. In and of itself I don’t have an issue with this statement, but it’s so broad as to defy any reasonable dissension. And that’s the issue I have with the term “test oracle.” It’s such a broad term as to be effectively useless since it not only uses itself to define itself, it confuses function and definition. I dug further into Mr Kaner’s definition where he outlines three capabilities of a testing oracle, and they’re all like that. It’s like if some one asked you to define a stop sign and you simply said, “It’s a sign that tells us to stop.” So any sign telling us to stop is a stop sign, even if it could reasonably be called something else such as a stop light or a crossing sign.
I stand by my statement about assumptions because the oracular definition of itself contains unstated requirements for assumptions. Mr Kaner’s definition, for instance, of the three capabilities continually references “the predicted behavior” but does not specify from where the predictions are generated. There has to be an initial assumption of behavior for a prediction to be made. Reading through other definitions from Hoffman to Memon, they all keep using the phrase “expected results.” It looks like everyone has decided using the (to my opinion) more appropriate/descriptive term “assumptions” is wrong so the term “expected” will be used in its stead.
I still wish to suggest we not use the phrase “test oracle” as it does not really describe what we’re attempting to accomplish with the overly broad term. I’d like to suggest we instead use any of the following phrases since they are generally more appropriate to what we are meant to be doing.
Testing augurs – Augury was a form of divination that identified and ranked the likelihood of several possibilities using a physical medium such as entrails or the flights of sacred birds
Testing sortes – Sortilege was a form of divination relying on the imposition of order on seemingly random occurrences such as the pattern of shuffled playing cards or cast bones
Testing omens – This form of divination simply required the recording and interpreting of unexpected or unusual events, the birth of unusual livestock being the most popular
I really like interviewing if its done well. The reason I like it is a good interview feels like an interesting conversation more than a date with a person you met online who you could swear said was taller than they appear now. A good interview sparks debate about subjects about which you feel passion. You should end up learning something or walking away with a few points or references you mean to research as soon as you get home. I prefer these sorts of interviews because it really lets both of us in on how it would be to actually work together. Answering a couple of technical questions the interviewer probably Googled up themselves fifteen minutes prior only proves the interviewee was lucky enough to get a question they know. It’s like the contestants on the game showJeopardy. Winning the day’s contest simply means you were lucky and either got categories about which you knew something or that your competitors were slightly more dense that you that day. It also means you remembered to phrase your answer in the form of a question, which actually comes up during bad interviews more often than one would think. The only way winning at Jeopardy proves you are smart is if you win consistently over time against a variety of other winners.
I bring this up partly to wedge another pop culture reference into my blog (and thus hopefully drive up my page views) but also because I had one of those good interviews a few weeks ago. Well it wasn’t really so much an interview as a questionnaire they sent me to fill out. To end the suspense, I didn’t even make it to a technical phone screening because I don’t have any Android or Windows Mobile experience. But I once again digress. The questions were nearly all those simple seeming one sentence things that end up requiring an answer in book form to address completely. The one I’d like to discuss further here simply asked what I thought were the qualities of a good automated test.
Good question, eh?
It surprised me how quickly I was able to respond, which tells me my trusty subconscious has been mulling over this for a while. Let me begin by referring you all to the eminent Michael Bolton’s answer for the nature of automation and why it is not “testing” so much as “checking.” As usual he makes a very compelling case in approximately a thousand words or less, so I’ll simply let his statements stand and pretend you’ve all clicked on the link and read it already.
So now that we’ve established what automation can and cannot do (again, click the link if you skipped ahead … and shame on you for skipping ahead) I’d like to make the case for better practice when designing an automated check or test case.
There are five specific primary attributes of an automated test case and so far five secondary attributes. The five primary attributes are the setup, the conditions, the check, the tear down, and the log. The five (so far) secondary attributes are some sort of common logging mechanism and repository, a common data library, some sort of common runtime engine separate from the application under test, and a queuing mechanism for sorting, organizing, and scheduling test execution without direct human initiation. I’ll expand on the primary attributes for the purposes of this blog. Let me know if you’d like to discuss the secondary attributes later.
In order to run an automated test/check, we first need something to check or test. This is realm of the setup attribute. Setup encompasses anything we need to do in order to arrive at a state where we can reasonably test. This may include everything from navigating through a user interface up to completely rebuilding an entire IT ecosystem from scratch. In some cases the teardown and setup attributes can get blurry if checks are included to just make sure things like test data and environments are the way we expect, but I’m splitting them out for the sake of cohesion.
After we’ve set up our environment, we need something to verify either positively or negatively. I call this primary attribute the conditions. Conditions are the patterns or actions we’d like to actively verify. The active portion is very important as automation cannot passively verify anything since it is only a check. In manual testing the conditions are usually used to define and organize the testing. Exploratory testing is primarily a condition set with no checks since the checks are experienced and cannot be known ahead of time. In terms of a simple manual test, the conditions would read something like “enter the text string ‘WWWWWWWWW’ and press the submit button.”
Which brings us to the actual check. The check attribute is the binay/boolean value the computer uses to mark a test cases as passing or failing. The check is the heart of an automated test/check and like a heart it works best when there is only one. It’s crucial an automated test only have one check because the point of automation is to limit the need for human involvement as much as possible. If a test case has fifteen checks, it then requires a great deal of human involvement to reason out which of the fifteen checks failed (if not all of them) when a failure status is registered.
After the check has been performed, a good automated test cleans up after itself with a teardown or “garbage collection” attribute. Better practices separate out the clean up from the setup in order to take advantage of reuse and aid in reverse engineering, but often you will see a sort of built in fail safe mechanism in the setup. This mechanism checks the state of the environment and tears it down before building it out if the prior test did not clean up after itself. While this redundancy reduces your risks for false failures, it does sacrifice performance, reuse, and scalability. It’s better practice to break these two actions out so you can call them separately at will.
Finally all messages and statistics judged meaningful should be recorded somewhere using the log attribute. The only required message is the state of the check, or the “pass/fail/not run” status determined by the binary check attribute. A case could be made for a historical log not being necessary, but I don’t personally believe this is to be valid. Remember the goal of automation is no human involvement during the test run, which I take to include monitoring the tests in order to record if the entire suite. If a test was run in a complete atomic vacuum, then a simply output window would suffice. Automation is nearly never run in this manner, however. This means we’ll need a physical log somewhere we can review at our leisure.
So that’s my thesis in a nutshell. There’s another question I culled from that interview I’d like to address, since it sent me down a very profitable pathway. But that’s for my next entry.
