Conducting a diary study to understand user behavior

Considering using a diary study to get to know your users, their goals, preferences, and behavior better? Then look no further.

In this article, we’ll run you through everything you need to know about this handy UX research methodology, including a hands-on guide to conducting your first one.

Here’s what we’ll cover:

Ready to get started? Then let’s go!

What is a diary study?

A diary study is a popular methodology used by UX teams that provides a record of the thoughts, experiences, ideas, and activities of users over a specific time period. Asking users to self-report in this way provides UX designers and researchers with a longitudinal record of users’ preferences and behaviors. This log can then be analyzed by the team for a more accurate understanding of their users’ overall experiences with a product, feature, or service.

Why use a diary study?

There are numerous advantages to conducting a diary study. As well as being an option that requires fewer resources than other UX methodologies, diary research provides real-life context to the problem you are trying to solve. It does this because your users are giving you specific details in their own words about how they are trying to solve the problem posed.

Diary studies offer UX researchers a flexible research methodology that can be used to help solve a variety of different issues. Whether you’re investigating what your competitors are up to, looking to improve your existing products, or simply hoping to better understand your current users, diary studies will provide valuable insights straight from the source.

Diary study research can also be a useful tool when your first early-stage prototypes are ready to be tested. You can assess the current popularity and success of the design with your users, as well as swiftly pinpoint any challenges they face, even at this early stage. They may even express in their diary entries their own ideas about how the design could be improved.

When your product has been developed, a diary study offers valuable insights into the user’s experience with your product in its final form. You can assess how well user expectations are met, uncover previously unseen issues, and identify any opportunities to improve the product based on this detailed user feedback.

Post-product launch, your users’ diary entries can be used to understand how well the product is functioning in a real-world setting and if any further adjustments or improvements could or should be made to future versions.

Types of diary studies

Before you start your diary study research, you’ll need to choose which type of diary study works best for the issue you are trying to get more data on.

Open diary study

As a free-form type of journal, an open diary study offers users a good amount of flexibility and freedom in how they record their experiences. Users can use the diary as they see fit, recording the thoughts or interactions with the product that they consider to be useful or of greatest importance.

While the UX team will provide some initial guidance in terms of, for example, how often to write in the diary and what information or thoughts users should consider including, generally the open diary format seeks to inspire free-flowing thought and expression.

Pros of an open diary study

An open diary study is a great way of introducing less-experienced users to a UX research study because it requires very little knowledge of UX or product development from those taking part.

In addition, users are likely to record information about their experiences with the product, feature, or service that the UX team have not previously considered, which is a great way for the UX team to learn more about the user’s experience and how they might improve it.

Cons of an open diary study

One of the disadvantages of conducting an open diary study is that the UX team may not get all the information or details they actually need from participants. Additionally, they may get a lot of information that they don’t need at all. This can mean that time is taken up reading through diary entries as the team looks for the entries which are relevant to the study.

Closed diary study

In contrast to an open diary study, a closed diary study is a more structured approach to this research method, providing users with a fixed set of questions to answer every day (or however regularly they are required to write in it). A closed diary study may also include specific instructions from the UX research team that the user needs to follow.

Pros of a closed diary study:

Data collected during a closed diary study is significantly easier to analyze and assess than the data from an open diary study. The preformatted questions and instructions of a closed study mean the team is only getting information that is relevant to the research.

Because the data received from participants is also in response to the same set of questions, the UX team can also much more easily compare and contrast answers between different users.

Cons of a closed diary study

Unlike an open diary study, the fixed nature of a closed diary study often means that researchers miss out on hearing about challenges or insights which the user hasn’t been specifically asked to comment upon.

How to conduct a diary study

While one of the advantages of conducting a diary study over other kinds of UX research is that it can be a fairly hands-off process for the UX team, there are still some key elements you’ll want to have in place in order to ensure it runs successfully.

Let’s take a look at what you need to plan and manage a diary study in order to get results.

Planning your diary study

For the UX team, most of the work that goes into a diary study is in the planning of it. When you begin your first diary study, you’ll need to determine your objectives, decide which type of diary study you’d like to conduct, write your users’ questions, and plan a start and end date.

1. What are your objectives?

In order to determine the goals of the research, you’ll first need to decide what sort of information you’re hoping to extract from your diary study participants:

• Do you want to understand how specific interactions with your product or service occur?
• Or do you want to know how the user feels while performing those interactions?
• Is there a new feature you want to test with users?
• Are you wondering why so many users abandon their shopping carts before making a purchase?

Establishing clear objectives now will guide and inform the research questions you’ll need to prepare. You’ll also want to think about whether you cover multiple topics in this diary study or focus on just one area.

2. Which type of diary research?

Next up in the planning phase is deciding which type of diary research you’d like to conduct. Are you hoping to get a more general feel of your product from an outside perspective, or do you need specific questions answered regarding a challenge or recurring issue?

Depending on these variables, you’ll want to choose either a closed diary study or an open diary study. (Some UX teams opt for a combined approach in order to get the most out of the research.)

Quantitative vs. qualitative data

When considering which type of diary study to conduct, it’s worth taking into account the specific type of data you’d like to collect; quantitative data or qualitative data (or both).

Quantitative data is measurable, numerical data, which can be used to identify the statistical significance in your findings. This type of data can be collected using close-ended questions, a Likert scale (where a user chooses a number on a scale to represent their answer, e.g,, 1=very bad and 5=very good), or questions with multiple-choice answers.

If you’re looking to gather quantitative data and get clear, measurable answers to specific questions, then a closed diary study is probably the best fit for your needs.

Qualitative data, on the other hand, provides researchers with more context and information around user behavior. It is non-numerical data that provides deeper insights into individuals’ beliefs, preferences, and understandings of a particular situation.

If qualitative data describes the sort of information and insights you need from your research, then it would be advisable to go for an open diary study.

3. Write your research questions

With your objectives pinned down and your diary style chosen, writing your research questions should be a fairly straightforward process at this stage in the planning process. Open diary study questions you might consider including are:

• How did it feel interacting with [product name]?
• What was your favorite thing about using [product name]?
• What was the biggest challenge you faced using [product name]?
• What would you change about [product name]?
• Describe your interaction with [product name] from start to end
• How did using [product name] compare with when you used it yesterday/last week/last month? Was it easier or harder?
• How did [product name] compare to similar products you have used? Was it better or worse?

Closed diary study questions might look more like this:

1=Very difficult; 2=Difficult; 3=Not easy or difficult; 4=Easy; 5=Very easy

• Using the scale, how would you describe your overall interaction with [product name]?
• Using the scale, how would you describe navigation between your cart and the checkout?
• Using the scale, how difficult or easy to understand was the on-page text?
• Using the scale, how difficult or easy was it to complete a purchase?
• Using the scale, compared with yesterday, how easy or difficult was it to change your profile picture?

4. Decide on a timeframe

The timeframe of your study will really depend on your overarching goals for the research as well as how much interaction with the product is required to get the data you need.

While some diary studies might take place over the course of a few days, others can take months. You’ll also need to gauge how much time a participant will need to set aside each day in order to complete the diary entries.

5. Select and recruit participants

Selecting and recruiting the right participants for your diary study will be a determining factor in its overall success.

You’ll be looking for individuals who can and will respond promptly to communication from the UX researchers and answer questions with the amount of detail you need to understand their perspective fully. Your participants will therefore need to be people who are keen to express their thoughts and happy to share them.

As we have seen, there can be a considerable and sometimes long-term time commitment involved in becoming a participant in a diary study. It’s important that you communicate this with potential participants before they agree to take part so they can see if this works with their own schedules.

In addition, the UX team will need to be clear with interested parties about what exactly will be expected of them on a day-to-day or week-to-week basis so that all expectations are aligned before the study begins and those who cannot commit can opt out in advance.

Another aspect to consider when recruiting for your diary study is the format by which users might be asked to give their responses. Some participants might prefer to write down their answers by hand, others might wish to log theirs online, and others might be used to recording video-based responses.

If your team has a preference for how they wish to collate data, express this to interested participants during the recruitment process; if an individual is only comfortable writing their answers by hand, but you need them logged in an online format, then you might have to reconsider recruiting them.

Some UX teams choose to interview potential participants as part of the recruitment process to assess their suitability for the study. This can be useful for gauging an individual’s commitment to the project and ensuring that they have a thorough understanding of expectations before signing up.

If you’re hoping for participants to contribute their answers to the study via video, an interview can also go some way toward informing the team of how clearly or thoughtfully the potential participant expresses their opinions.

If you plan on using incentives to motivate participants, don’t forget to be explicit from the start of the recruitment process about what these will be and when the participants can expect to receive them.

Commencing the diary study

So you’ve decided on your objectives, chosen a type of diary study, written your research questions, nailed down a timeframe, and, finally, selected and recruited your participants: it looks like you’re good to go!

Once you’ve sent out the questions along with some guidelines regarding what you’re expecting from the diary entries, you’re going to be pretty hands off during this research study.

Although a diary study involves considerably less involvement than other types of UX research, there will still be some small administrative tasks to attend to while the study is running.

For example, you may choose to set up reminder emails or notifications for your participants to ensure they stay on track with their diary contributions. Additionally, it’s probably wise to check in with them every now and then to make sure everything is running smoothly. Finally, you should be prepared to answer incoming queries from participants who may need clarification of what’s expected of them, or are having technical issues.

It’s important to recognize the effort and contribution your participants are making to be a part of the study, so be sure that any communication you have with them is friendly and appreciative. Keep an eye on individuals who are struggling and respect each person’s boundaries.

Have you chosen to have entries be sent to you as they are logged, or will you receive everything from participants at the end of the dedicated time period? If you’ve selected the former, take notes and process the entries as you go, so as not to get overwhelmed.

If you’ve selected the latter, consider using a smart diary study tool to help you organize entries. These can often assist in other areas too, such as sending reminders, and recruiting participants. My favorites are: dscout, Indeemo, SurveySignal, MetricWire, and QuestionPro.

After the study

After the study has concluded, ask participants for their feedback on the process. You may also want to conduct post-study interviews to get more detail on top of their diary entries or to clarify specific points.

Data collection and analysis

If you’re manually collecting and analyzing data rather than using a diary study tool, your first step will be transcribing your diaries and logging entries in one place, such as a spreadsheet.

Once all of your entries are in one accessible place, you’ll need to start sifting through the data and looking out for patterns and trends. From here, you can begin to put together your first hypotheses.

At this stage, it can help to pinpoint strong participants who have submitted data that is particularly thoughtful, detailed, or thorough. This is because the more robust the information from one candidate is, the easier it will be to identify patterns within their entries. Once identified, these patterns can then be supported with evidence from other participants whose entries might be slightly less detailed.

Tagging your data with the correct labels is another important aspect of data collection and analysis. This will save you a considerable amount of time when trying to locate key pieces of information later on and make it much easier to spot patterns and trends.

Best practices for diary study analysis

Consider using a tool

While one of the benefits of a diary study is that it requires relatively few resources when compared to other types of UX research methods, it’s not a bad idea to consider using a free or paid-for diary study tool.

These days, digital diary study tools can assist throughout the entire process, from helping you define your objectives and write your questions to recruiting participants and organizing and analyzing your data.

Although analog is always an option, digitizing the process will streamline activities and save you time and stress in the long run.

Onboard participants

If you want to ensure the success of your diary study, onboard your participants properly! This can be done with a short briefing document that is sent to participants along with the questions, or, even better, done via a short video call. The more prepared your participants are, the higher the quality of answers will be that you’ll receive.

Stay in touch with participants throughout

It should go without saying, but you’ll want to check in on your participants over the course of the diary study. Depending on their preferred method of communication, make sure they know you’re available to help out if there are any hiccups or if they simply have a question about the process.

Consider using incentives

For some shorter studies, participants are happy to help out without being incentivized, but for longer studies it makes sense to offer participants incentives to keep them motivated and to encourage high quality contributions. This UX research incentives calculator is a useful tool for calculating exactly what to offer.

Conclusion

We hope you’ve enjoyed learning about diary studies in this article! I’ve covered what a diary study is and why it’s a valuable UX research method as well as provided you with the steps you need to plan and conduct one successfully.

Want to know my top tips? Here they are!

• Make sure you select the diary study type that best suits your needs; if you need detailed, nuanced insights about the user’s experience with your product, opt for an open diary study. If you need measurable, comparable stats or figures, a closed diary study will better serve your needs.

• Finally, keep the lines of communication open between the UX team and your participants; you may need to tweak the process along the way, and the more available you are to participants the easier that will be. Good luck!

Header image source: IconScout