50.110
Information Discovery
Martin Schedlbauer, PhD
2024-02-14
Objectives
Upon completion of this lesson, you will be able to:
- set up and conduct interviews
- discover information processes using observation
- leverage surveys
- conduct document analysis
- understand information exchanges through interface analysis
- list various biases
Overview
Elicitation is the act of obtaining information from the stakeholders, knowledge holders, and subject matter experts. It is essentially a fact-finding mission where the insights of domain experts are uncovered. Elicitation requires a significant amount of personal interaction and collaboration between the analyst and the knowledge holders.
All elicitation sessions ought to be held as collaborative sessions, ideally as face-to-face meetings. When direct contact with the experts is not possible, then video conferencing, online conferencing, or telephone conversations can be substituted, although these communication modes are not nearly as efficient as direct interaction.
The goal of information elicitation is to discover facts and express those facts as information objects, properties, relationships, constraint, statements, rules, and processes.
Elicitation Techniques
There are a number of commonly used elicitation and fact discovery techniques, including brainstorming, interviewing, discovery prototyping, facilitated workshops, user task analysis, observation (job shadowing), document analysis, interface analysis, reverse engineering, and surveys. Not all are equally important. The most important ones are brainstorming, document analysis, interviewing, observation, interface analysis, and survey. Those are presented in this lesson.
Information Discovery Process
In most situations, information is discovered and then analyzed and modeled in a visual language such as the Unified Modeling Language (UML). Next, the information is documented externally in a shared information environment so that it can be communicated to interested parties. Finally, it is validated to ensure accuracy, currency, and validity.
Recording Discoveries
Any insights or discoveries made during elicitation sessions must be recorded, first informally and then, eventually, formally. There are a number of approaches to recording information object, properties, structure, and facts:
Write discoveries on index cards or sticky notes and pin them to a whiteboard, wall, or on top of a table. Index cards are particularly useful because they can be moved around.
Use whiteboards or flip charts to record your discoveries.
Record the elicitation session so that details can be revisited later. Be sure to check with the session participants whether they agree to being recorded. Keep in mind that they may not be as honest if they know they are being recorded.
Use computer-based visual modeling tools to record process insights. While this approach generally yields the cleanest output it is often the least effective as there is often too much of a focus on the workings of the tool rather than the interaction between the information analyst and the knowledge holders
The use of tangibles such as index cards or sticky notes is encouraged during elicitation. Give session participants their own pen to write and let them get involved. Engaged participants contribute better insights and they understand that they own the information or processes being modeled. Tangibles on walls or tables can also be moved around more easily and information objects and process steps can be more easily grouped and ordered.
While computer-based modeling tools aren’t bad per se, they do have the drawback of making session participants more passive. Someone has to “drive the tool” which means that the others are now passive observers and will likely feel less engaged and may become distracted.
Brainstorming
Brainstorming is used to discover information objects, properties, structure, ideas, novel concepts through informal (but structured and facilitated) group sessions. A good facilitator is essential to the success of such sessions. Facilitation aids such as mind mapping, Ishikawa diagramming, or context diagramming are commonly employed during brainstorming sessions to increase their efficacy. Brainstorming is almost always followed up with more in-depth elicitation techniques such as interviewing or observation.
Brainstorming is about generating ideas and thinking creatively. It is intended to exploit associative thinking in which an idea can lead to many other ideas.
During the brainstorming session, the facilitator should record the ideas on a whiteboard, flip chart, a software tool, index cards, sticky notes, or one of the diagrams below.
Sessions should be short (30-45 minutes) and have a small number of participants. Groups with more than 5 participants tend to not be effective. Don’t select more than 3 to 5 participants for a brainstorming session. If there are more, split the group into multiple subgroups. Focus on generating lots of ideas; no idea is bad and you should focus on quantity not quality. Don’t criticize ideas and encourage new ideas to be generated from existing ones, i.e., associate freely.
If you see that a participant is dominating the discussion or that some participants feel intimidated by others, have the participants write their ideas anonymously on index cards and then pin them up for discussions. Read them out loud and debate them. Generate new ideas through verbal interaction or additional index cards.
You can also create an electronic brainstorming chain by sending an e-mail to the first person in the brainstorming team with a seed idea to explore. He or she then comments on the idea and sends it on to the next person. The next person responds and sends it on again. This process continues until no further ideas are added to the e-mail chain.
It is best to set goals during brainstorming sessions. Ask participants to come up with a specific number of ideas, or add some minimum number of additional ideas onto an existing concept.
There are a number of tools that can aid in organizing and focusing a brainstorming session. Among these tools are:
- Mind Mapping
- Ishikawa (or Fishbone) Diagramming
- Event Mapping
- Context Diagramming
- SWOT Analysis
Interviews
Interviews are a direct interaction (face-to-face or via communication technologies such as online meeting spaces like Microsoft Teams, Zoom, or Google Meet) between the information analyst and the knowledge holders, domain experts, and other stakeholders. Interviewing is probably the most widely practiced elicitation technique and is often used in conjunction with observation and after initial brainstorming sessions.
Interviews are used to find, verify, and elaborate facts as well as identify information object and their properties, and process steps. They are also employed to solicit ideas, opinions, and preferences. Interviews are also useful to generate enthusiasm about a project or initiative and to get stakeholders engaged.
Ideally, interviews are conducted as face-to-face sessions lasting about 30 minutes to an hour, with 45 minutes being the ideal length. Longer interviews are discouraged as participants tire after about 45 minutes. If a face-to-face meeting is not feasible due to time or geographical constraints, then an virtual meeting can be substituted. However, unless video is used, gathering facts through technology-mediated communication can be risky as emotions, facial expressions, and non-verbal communication are not observable.
Interviewing Guidelines
Initially, an unstructured interview – conducted similarly to a brainstorming session with few, if any, specific questions – may be appropriate, but then a more structured interview with a specific set of questions should be used. You can start by asking open-ended questions requiring a free form response but then follow-up with closed-ended questions where you expect the interviewee to pick a response from a list of specific choices or respond with short and direct answers.
Gauging a person’s communication style might be beneficial. Find out if the person is an effusive talker or needs encouragement and specific questions to reveal facts. Adjust your questioning accordingly. Interact with interviewees to have them respond freely and openly, but probe for more feedback. Observe nonverbal communication; it can reveal a great deal about a participant’s attitude toward the project or line of questioning.
Interviews are an effective although time-consuming way to get facts. Their success is highly dependent on the interviewer’s human relations skills. Make sure that you avoid loaded or leading questions that might reveal your own bias. Don’t include your opinion as part of the question. Use clear and concise language and avoid long or complex questions. Avoid using technical jargon that might not be understood by your interviewees.
In the interview listen carefully, maintain control of the interview, ask probing and follow-up questions, and make sure that you understand what the interviewee is saying. Repeat what they are saying and practice active listening. You should listen more than talk.
Observe their mannerisms and nonverbal communication; it might reveal a lot about how they feel about the project effort, you, or your questions. Above all, be patient, keep the interviewee at ease, and maintain self-control. Be sure that you finish on time – stick to your deadlines; after all, your stakeholders may have another commitment right after your interview; value their time.
When possible bring a scribe to take notes or record the meeting using tools such as LiveScribe, Google Keep, OneNote or Audacity – these tools allow archiving and searching of the recording for keywords later. Of course, ask for permission first before you record; it might be against an organization’s policies.
Interview Preparation
Prepare for the interview by writing down your agenda in an interview guide. In the guide, list the questions you expect to ask and the amount of time allotted for the question. Record the answers to your questions in a log. The guide can be a printed form or completed on a computer using a word processor or some other tool. Keep in mind that typing while someone is speaking is often perceived as distracting.
Always ask who else you should be speaking to when you conclude an interview; this reduces the likelihood that you forget to consult an important stakeholder.
Common Interview Questions
Below is a list of direct as well as context-free questions that can be helpful during interviews to elicit facts, information objects, properties, insights, and processes:
- Is there anything else I should be asking you?
- Are there others whom I should be talking to?
- What happens if <some event> were to occur?
- Are there any constraints, rules, or regulations that need to be observed?
- Do you have a current product, system, or process that you use? What do you like about it? What does not work? What would you change? Why?
- Does everyone use an object, system, or process the same way? Is there a standard procedure?
- Can you describe the usage environment for the information, process, or system?
Document Analysis
Document analysis is an inspection of existing documentation including organization charts, operating procedures, training manuals, databases, data models, information systems, user interfaces, flowcharts and other process maps, and any other documents that provide context and background to the analysis effort.
Observation
Observation is a fact-finding technique in which the business analyst either participates in or watches a person perform activities to learn about a business process or difficulties in using a system. This technique is also known as job shadowing.
Observations can be active or passive. In an active observation, the business analyst asks questions, probes for details, and perhaps even participates in the process. In a passive observation, the business analyst does not interact with the person who is being observed; ideally, the person does not even know they are being observed, although that’s often difficult to do in practice.
If a process or an activity has variations and is done differently at different points in time, then using work sampling – observations taken at random intervals – is a great way to cut down on the overall observation time. Use the sampling procedure presented below to help determine how many work periods you should observe to get an understanding of the variations in a process.
Benefits and Drawbacks of Observation
The chart below summarizes the benefits and challenges of the observation technique. Overall, observation is a great way to see how work is actually done rather than how stakeholders describe the process in interviews, although when observed some stakeholders may not do the work as they normally do – they may do it “by the book” and it’s hard to see what they actually do when the business analyst is not around.
| Benefits of Observation | Drawbacks of Observation |
|---|---|
| Actually see how work is done rather than how people describe the work in an interview or in an operating procedure | People may perform process differently when being observed |
| Complex tasks are easier to understand | Work observed may not be representative of normal conditions |
| Relatively inexpensive to conduct | Timing can be inconvenient |
| Interruptions can throw off time measurements | Some tasks not always performed the same way |
| Ability to take measurements of time needed or materials used | Different people may perform the same task differently |
| Easier to identify hand-offs between multiple parties |
Active vs Passive Observation
Observations can be active or passive. In an active observation, the analyst interacts with the subjct being observed while in a passive observation, the analyst does not. However, the mere presence of an observer may cause the subject to perform the process tasks differently (often called the Hawthorne Effect). So, passive observation is often also invisible where the subject does not know they are being observed. Not interrupting while observing makes timing measurements easier and has less impact on the tasks being performed but sometimes being able to ask question or get explanations about why some task is being done in some particular way can be useful. However, if subjects are being asked, they may inadvertently change their behavior or task sequence. It can also be uncomfortable or distracting for subjects to be questioned.
Challenge Questions
During an active observation, the business analyst should challenge the domain expert in order to elicit hidden assumptions and requirements details:
- Is the work always done like this?
- When would you do this differently?
- How often does this activity get done in a day (or hour, week, month)?
- Does everyone perform the task the same way?
- Who does it differently? Why?
- Is there information you rely on during this process? Where do you get it from?
- Do you produce any information? If so, to whom do you provide it?
- Are there any rules you have to follow? Who defines these rules?
- Is this a standard process? Is the standard documented? Where?
- What would happen if the process were done differently?
- Does it always take the same amount of time?
- What events could occur that would change the process?
Interface Analysis
An interface is a shared boundary between two components, processes, systems, humans, or organizational units. Most information systems require some kind of connection with other applications, as well as hardware and peripheral devices to function properly. Interface Analysis is a business analysis elicitation technique that helps to identify interfaces between system components to determine the information flows between the components. The simplest example of an interface is a user interface that defines the shared boundary between humans and an application or system.
Interface analysis is used to identify where, what, why, when, how, and for whom information is exchanged between system components or across organizational boundaries.
Interface analysis defines and clarifies the use of an interface, what information is being exchanged, when the information will be exchanged, where the information exchange occurs, and how the interface is implemented. It helps understand the information that is needed by a system or organization and the information that is generated and shared by a system or organization.
Context Diagrams are often used to visualize interfaces, system boundaries, and information exchanges.
Bias in Information Collection
Humans are subject to a variety of cognitive biases. Read the following articles on the web for an expanded perspective:
de Bono’s Six Thinking Hats
One technique that is often used in collaborative information collection sessions is Edward de Bono’s Six Thinking Hats Technique where each participant takes on a different role.
In this technique there are six different colored hats (plus, sometimes a purple hat, for the product owner):
White Hat: Listens, questions, looks for information that is needed but not yet available.
Red Hat: The emotional hat that allows for unconstrained thinking and gives permission to express hunches without evidence.
Yellow Hat: Thinks about benefits, savings, and advantages. How something might be good for the organization. Encourages optimistic and “sunny day” thinking.
Green Hat: Associated with creativity and new ideas. Thinks outside the box. Encourages proposals, suggestions, ideas, alternatives, provocations, and brainstorming.
Blue Hat: Thinks about the practicality if ideas and proposals. Not concerned with the ideas or information itself but the execution of processes and how to do the work.
Black Hat: Forces consideration of risk, reasons why something may not work, why it may be illegal or unethical. How actions could go wrong. Reasoning must be logical and grounded in evidence (otherwise it is emotional, that is_i.e.,, the red hat).
Use Edward de Bono’s Six Thinking Hats approach to force perspective and capture new information. You may assign a hat to the entire group or a different hat to different information elicitation session participants; perhaps rotating the hats to encourage brainstorming. On occasion it can be useful for all participants to “wear the same hat” so that everyone is thinking in the same direction. Of course, in some situations, some hats might not make sense, so do not feel compelled to use every hat.
Be sure that as a facilitator that you encourage input from “all hats”; perhaps rotate input and go in sequence: red, yellow, green, etc.
It is a good practice to assign hat and not ask participants which role they would like to take on. They do not know and rely on your expertise for the method. You should have some type of visual legend or responsibility summary to explain the perspectives/hats.
Lecture
Slide Deck: Information Discovery
Summary
Information discovery is an essential phase in virtually any information science effort or project. There are numerous techniques at the disposal to the information scientist with interviews, surveys, focus groups, observation, interface analysis, and document analysis chief among them.
All Files for Lesson 50.110
Errata
None collected yet. Let us know.