Keywords

1 Introduction

Software start-ups focus on developing innovative products or services, often disruptive and in challenging contexts [20]. A start-up is a human institution designed to deliver a new product or service under conditions of extreme uncertainty [21]. To be a start-up, it is not require the company to be small or new. Start-ups differ from established companies by searching for a scalable, repeatable, and profitable business model with the aim of growing in the market [22]. The continuous seeking for a scalable business model pushes the start-ups to have changes in their process and operations constantly. Start-ups usually work with a small team of professionals, exploring new technologies, being marked by rapid evolution, high uncertainty about customers and market conditions, and high failure rate  [1, 20]. The literature has emphasized the need for studies on Software Engineering practices that are specific and appropriate to the characteristics of start-ups [20, 22].

The combination of innovation and a market-driven context leads to a situation where a specific (and restrict) set of software development practices may be useful for start-ups [19]. Besides, some studies point out user experience (UX) as a critical factor for software start-ups, generating value for users, and creating competitive advantage [8, 12, 20, 22]. Although different UX definitions are found in the literature, it is a consensus that UX work can affect a product acceptance [14]. The ISO-9241 norm [2] defines UX as “user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service.” This norm emphasizes human-centred design, highlighting the adoption of different software development practices, such as user research, interactive design, user involvement, and cross-functional design teams.

For the Software Start-ups Global Research Network groupFootnote 1, good UX can help start-ups to move towards successful and sustainable business creation, promoting genuine interest from users and opportunities for meaningful feedback [22]. UX can be crucial to test a value proposition early on [6]. At the beginning of product development, a good understanding of the problem the software will solve can helps to find a suitable market and drive the UX work. To assess market interest and to establish a customer base, studies indicate that many start-ups attempt a balance between customer value by focusing on functionality and UX while keeping engineering effort minimal [12]. In this sense, a trade-off between features, quality, time, and cost can determine the scope of the minimum viable product [21]. However, there is a gap in knowledge about UX practical actions more compatible with reality in start-ups. To accomplish this gap, we carried out a survey in Brazil’s start-up ecosystem to identify how UX work has been applied in these fast-growing innovative environments. In this paper, we present our findings on UX relevance from a software professionals’ standpoint and identified a set of challenges faced for the adoption of UX practices taking into account the reasons, obstacles, and skills needed for UX work.

Our survey’s target audience was professionals working with software development in start-ups, covering start-ups of any size, lifetime, and market segment. Based on 88 valid answers from professionals who work on software development positions in start-ups, our results offer a diagnosis about UX, revealing that although it is considered relevant and present, it remains often in the frontiers of the software development process and not integrated into it. Our work contributes to research and practice of UX in start-ups, both by bringing more discussions about the critical role of UX in the different phases of software development, as well as by showing that how much have or not have UX positions allocated in software start-ups can influence the use of UX in the practice. Additionally, six challenges related to UX adoption were identified, providing important insights to further academic and practical studies in the software start-ups context.

This paper is organized as follows. Section 2 introduces the related works to UX in start-ups. Section 3 outlines the research method adopted. Section 4 presents the main findings and discusses the study limitations and threats to validity. Section 5 highlights the challenges that start-ups face on UX work. Finally, Sect. 6 concludes the paper and presents directions for further work.

2 Related Work

UX’s importance in software development in start-ups was raised in the literature in 2012 [17]. Five recommendations about software engineering practices for developing start-up products are reported, and Good UX is essential recommendation warns on risks of skipping UX validation in the early stages of the product conception. In 2016, a global research agenda on software development in start-ups restated the importance of put UX in practice from the early stages of software development [22]. However, this work pointed out that it is still an open question which UX practices are in fact useful for these organizations.

Although software practitioners in start-ups recognize UX’s relevance to product development, start-ups have faced difficulties in setting the UX methods and practices to the fast-paced environment [6, 7]. The software teams’ mindset and organizations’ culture can often be an obstacle to UX work [16]. There is also an influence of the limitations in human resources, common in most start-ups, which can make it difficult to hire experienced UX professionals [9, 12, 13].

Collecting users’ feedback about the product appears as the main concern of start-ups [7, 8, 12]. User interface prototypes (i.e. mockups and wireframes) [9, 12, 17], user testings [9], questionnaires [13] and analysis of data log [7, 8] are mentioned as the UX practices frequently used by software practitioners in that context. However, start-ups also struggle with the effective use of information collected about their users [5, 8, 9]. Moreover, UX methods and practices have different aims depending on the software development stage [12], requiring UX-skilled professionals to know which one to use and how to get the best benefit from their application. Some studies stand out that the relation between UX and business models could support decision-making about product development and contribute to a positive image of the organization [9, 12, 22].

From the literature review, we identified only two surveys focused on exploring the topic of UX in start-ups [9, 15]. In the survey conducted from interviews with 21 professionals of 20 software start-ups from different countries, Hokkanen et al. [9] identified three main categories of factors that affect UX work in start-ups were identified: (1) Strategy, as strategic choices on resource allocation and Product qualities that affect actions for creating good UX; (2) Team Qualities, as they can be improved by having UX expertise, domain knowledge, and UX mindset; and, (3) Interaction with Users that focus on how actively involving users in their process of creating UX. Salgado et al. [15] conducted a survey involving 26 small Brazilian software companies of which 17 were software start-ups. Their survey results pointed out (1) the need of improving the professional awareness of the importance of usability and UX, (2) a demand for more appropriate usability and UX practices to fulfill the context of small businesses.

3 Research Method

Our research method followed the guidelines recommended by Kitchenham and Pfleeger [11]. This survey addressed two research questions. First, what is the role of UX in the different software development activities? Second, how do professionals consider UX in start-ups that have UX-related roles and those that do not? By answering the first question (RQ1), we intend to explore how often UX activities have been carried out in the start-ups throughout software development, and what UX practices have been employed to support the professionals’ tasks. To address the second question (RQ2), we intend to investigate how professionals perceive UX work in the start-ups by exploring the reasons that drive and factors that hinder the use of UX practices in the start-up. Furthermore, we look at UX work relevance and essential related skills in software start-ups from the point of view of professionals who work in the UX area in comparison with those professionals with positions only in the software engineering area.

The survey’s target audience was professionals working in software development areas at start-ups in Brazil, including software engineers, software developers, UX designers, testers, project managers, product managers, and software architects. In this survey, we do not impose any restrictions on the start-up size. As the data collection method, we prepared an online survey with 26 questions, including an informed consent describing study objectives and a statement about the guarantee of participants’ confidentiality and anonymity.

The questionnaire was evaluated in two steps. First, three researchers of UX, software engineering (SE), and start-up areas reviewed the survey design. Second, we conducted a pilot test with six graduate students from UX and SE areas to verify the questionnaire clarity and easiness of understanding. Our instrument was refined based on that feedback by: (a) rewriting questions; (b) modifying answers’ format; (c) reordering questions to a more logical sequence; or (d) splitting questions to improve readability. Despite the survey’s broader scope, we used a subset of questions related to this paper’s aim, presented in Table 1. The online survey was open from June until the middle of September 2020. An invitation email was sent directly to start-ups and researchers’ contact networks, asking to forward it to other potential participants. The respondents were guided to answer considering the start-up where they currently work.

Table 1. Survey questions
Full size table

In our analysis, we first verified responses’ consistency and integrity, eliminating inconsistent, incomplete, and duplicate questionnaires. In the next step, we did the partitioning of the responses (see Table 1). To analyze the responses related to RQ1, we did a first division based on the respondent’s information on how long the start-up has been using UX practices (Q7). Regarding responses related to RQ2, we did a second division based on the respondent’s information about whether the start-up had UX positions (Q11) to analyze the responses from Q12 and Q13. To analyze how much certain skills were necessary to perform UX activities (Q14 and Q15), we did a third division, considering whether the respondent plays a UX-related role or not (Q2). Finally, we considered the last two divisions to analyze how relevant is UX to the development team evaluating the influence of having a mindset focused on programming (Q16).

We analyzed the data using descriptive and statistical methods. A non-parametric test Kruskal–Wallis (KW) was applied to verify statistically significant differences between observed groups [18]. For cases where the KW test showed a significant difference, we ran a post hoc analysis using the Dunnett’ method [3] to identify the differences.

4 Results

A total of 99 responses have been collected, out of which 88 have been used in our analysis as valid responses. Invalid responses refer to 9 duplicate answers, 1 with incomplete answers, and 1 with inconsistent answers. Of the 88 respondents, about 80% of them have more than 3 years of professional experience, and out of these, 29 respondents have more than 10 years. About 51% of respondents (45 of 88) play roles only in the software engineering area, 23% of respondents (20 of 88) play roles only in the UX area, and 26% of respondents (23 of 88) accumulate positions in both areas.

Of the 43 respondents who informed work on UX-related roles, the main reported roles were UX Designer (38–88%), and UX Researcher (21–49%). Regarding SE-related roles, we mainly found Software Developer (33 of 88) and Software Architect (16 of 88). About 40% of respondents played the role of product manager (35 of 88) and 23% product owner (20 of 88). The respondents’ start-ups develop products for different market segments, and some of them for more than one segment. The three most cited start-ups segments are Information Technology, Finance, and Education.

Figure 1 introduces a heat map chart presenting the number of start-ups that fit into each category related to some characteristics, such as size (in terms of employees), foundation year, and the amount of time start-ups have been applying UX practices. Lighter colors represent that few start-ups fit into those categories, whereas darker colors represent that plenty of start-ups fit into those categories.

Fig. 1.
figure 1

Start-up time applying UX practices by size and foundation year.

Full size image

Regarding the frameworks and methodologies applied in start-ups (Q6), the participants were allowed to select more than one option. SCRUM (55–63%) and Kanban (54–61%) are the most recurrent among respondents. Only respondents who work at start-ups that apply UX practices (78 of 88 respondents) mentioned Design Thinking (41–53%), Design Sprint (31–40%), and Lean UX (25–32%). The least applied methodology is Hypothesis-driven development (5–6%).

Table 2. UX in the software development
Full size table

4.1 UX in the Software Development Activities

We asked the participants how often UX is applied throughout software development (Q8), how often activities involving users occur (Q9), and which UX practices were commonly used in the start-ups (Q10). Of the 88 valid responses, 78 respondents answered these questions, while the other 10 respondents reported that their start-ups did not use UX practices. To analyze these questions, we split the 78 participants’ responses according to information about how long the start-up has been using UX practices (Q7). We found that 18 respondents were from start-ups that have applied UX for up to 1 year, 32 respondents from start-ups that have applied UX from 1 to 2 years, and 28 respondents from start-ups that have applied UX for over 3 years.

UX in the Software Development. In Table 2, we see the respondent’s perception of how often the start-ups are concerned with applying UX in the software development. Overall, almost 90% of respondents stated to use UX practices more often during product specification, design, and prototyping activities, while more than 40% of them rarely apply it in the software maintenance. We found that more than 80% of start-ups working with UX for more than 3 years are significantly more concerned with it during the software ideation (p-value 0.041) and requirements analysis (p-value 0.038) activities than the start-ups that started using UX less than a year. Besides, more than 80% of start-ups that have applied UX for more than 3 years are significantly more concerned with involving users in eliciting requirements (p-value 0.022), applying UX in testing and integration (p-value 0.067), and maintenance (p-value 0.033) than the group of start-ups using UX from 1 to 2 years. In the product specification phase, we found that both groups with applied UX for less than 1 and 2 years differ significantly from the group that has applied UX for longer.

UX Practices Applied in the Start-Ups. Table 3 presents the respondents’ information about which UX practices are used in their start-ups, as well as their frequency. Overall, the most commonly applied practices are prototyping (66 of 78), user interview (63 of 78), and usability testing (51 of 78). Heuristic evaluation (28 of 78), storyboard (26 of 78), and card sorting (19 of 78) are the least used practices in start-ups. KW test revealed a statistically significant difference in four practices: user interview (p-value = 0.05), usability testing (p-value = 0.006), personas (p-value = 0.007), and storyboard (p-value = 0.004). The post hoc test indicated that start-ups from Up to 1 year group use significantly less of these practices than the star-ups from over 3 years group.

Table 3. UX practices applied in the start-ups
Full size table

Figure 2 shows the UX practices used according to the foundation year of the start-ups. From the heat map chart, we can see that the newer companies (i.e. <3 years) use more personas technique than user testing. This result shows that these organizations need to have more details on the users’ characteristics. These organizations are in the early stage of product exploration, and, consequently, the users are not well defined, the product is not stable, and they are subject to pivot more. In older start-ups (i.e. >3 years), the testings are running more frequently than in newer ones, which can be a consequence of their product has already achieved a degree of maturity and been considered stable.

Fig. 2.
figure 2

UX practices applied in the start-ups.

Full size image

4.2 UX Work from the Perspective of Start-Ups Professionals

To explore perceptions about the main reasons why the start-up uses UX practices (Q12) and the main factors that hinder their usage (Q13), we divided the participants’ responses considering whether or not respondents’ start-ups had UX-related professionals (Q11). Of the 78 respondents who answered these questions, we found that 33 respondents were from start-ups that have a fully or partially dedicated UX team (UX-Team), 25 respondents were from start-ups that have at least one fully or partially dedicated UX professional (UX-Pro), 12 respondents from start-ups that do not have a UX professional (No-UX), and 8 respondents from start-ups that train employees from other areas to do UX work (Train-E).

Reasons for Using UX. Table 4 summarizes the results on the reasons that lead start-ups to apply UX practices in the development of their products. From the respondents’ point of view, the three main reasons for the start-up to use UX practices are to create value for the user (70 of 78), create successful products (66 of 78), and create value for business development (56 of 78). KW test reveled a statistically significant difference in respondents’ opinions on obtaining competitive advantages (p-value = 0.020). This reason was highlighted by 67% (39 of 58) of respondents from UX-Pro group. In the opinion of respondents from Train-E group, the two main reasons to apply UX are the successful product creation (7 of 8) and the value for the user (6 of 8), and the least cited reason is about understanding how to sustain the long-term business model (1 of 8).

Table 4. Reasons for using UX
Full size table
Table 5. Factors that hinder the application of UX
Full size table

Factors that Hinder the Application of UX. Table 5 presents respondents’ opinions about factors that make it difficult to apply UX practices. Overall, more than 40% of respondents pointed out that the main difficulties are the short time available (36 of 78) and the scarce financial resources (31 of 78). Unsurprisingly, lack of a designated professional to work with UX is a difficulty most often pointed out by respondents from start-ups that do not have UX professionals, while the difficulty related to communication and collaboration gap between UX and other professionals is a difficulty most often pointed out by respondents from start-ups that have UX teams.

Skills to Perform UX Activities. To explore how much certain skills were necessary to carry out UX activities from respondents’ viewpoints (Q14 and Q15), we analyzed 78 responses considering whether or not the respondents played the role of UX in the start-up (Q2). We found 43 respondents who play some UX-related role in their start-ups and 45 respondents who play roles only in the area of software engineering. See the results in Table 6. Overall, skills know-how to collect and interpret feedback and Know-how to collect and interpret user information were considered much-needed for the group of professionals who play the UX-related role (an average of 72% of them), and to a lesser proportion for SE professionals (an average of 54% of them). When running the KW test, we found no significant differences between groups.

Table 6. Skills to perform UX activities
Full size table

UX Relevance and Focus on Programming. The question about the importance of UX practices for the development team considering a mindset focused on programming from the respondents’ perspective (Q16) was analyzed in two ways. Table 7 shows on the left side, the results considering the relation of UX relevance and the UX positions in the start-ups (i.e., UX-Team, UX-Pro, No-UX, Train-E). On the right side, the table shows the results considering a mindset focused on programming. We found out that there are no significant differences between groups in both questions, according to the KW test. Regarding the importance of UX, only 13% (10 of 78) of the respondents responded that UX is rarely important for the development team. Development teams are always concerned with UX in the opinion of 46% (19 of 41) of respondents within UX-Team and Train-E groups. Table 7 also shows an overall perspective of the results by dividing the data into UX and SE professional groups. As in the previous analysis, we did not find significant differences between these groups for both questions.

4.3 Threats to Validity

The respondents’ perception and their inadequate knowledge of the domain are potential threats to the internal validity of surveys [4]. We conducted a pilot test with researchers in the UX domain and start-ups to check possible problems with the survey questions, and then, we refined the instrument to mitigate misinterpretations by respondents. The survey participants were recruited using a convenience sampling [4]. Regarding external threats referring to sample representativeness and heterogeneity within the target population [23], we use a network of start-up professionals and researchers distributed in different states of Brazil to engage professionals working for software start-ups and playing different roles.

Table 7. UX relevance and focus on programming
Full size table

We refined our questionnaire by changing some types of questions, such as adopting the multi-point Likert scale to mitigate possible construct threats referring to measurement fails. In multiple-choice questions, we add the option “other” to avoid an exhaustive list of possible answers. And, for all multiple-choice and Likert scale questions, we include the “I do not know” option to avoid inconsistent responses. Due to space limitations, however, we did not add the answers to these options in the tables of this paper. Specifically, the list of UX practices and the reasons and obstacles statements for using UX in the start-up were derived from existing studies [8, 9, 15]. To mitigate the evaluation apprehension effect, we have guaranteed the respondents anonymity and made the survey results available through a technical report. Regarding conclusion threats, we used the Kruskal-Wallis non-parametric tests and the Dunnett’ method that are appropriate to perform multiple comparisons from multiple choice answers or the Likert scale [18].

5 Challenges for UX in Software Start-Ups

Regarding RQ1, our findings revealed that in start-ups applying UX for over 3 years, UX is more present in the different software development activities, and the adoption of UX practices is less frequent for start-ups that apply UX for less than 3 years. This result suggests these start-ups become more mature in understanding the importance of UX and using UX practices in their work.

Regarding RQ2, our results revealed that independently of the start-up having UX-related roles, respondents considered the creation of value for the user and the business reasons for using UX. The respondents also pointed out the lack of resources and time as the main impediments that blocked UX adoption. Our findings showed that UX is considered relevant even in start-ups where there were no UX professionals. The ability to handle user information from different perspectives was seen as the primary skill to conduct UX work in start-ups. Taking into account our findings, we outline some challenges on UX work in start-ups as the following:

Challenge 1 - Matching UX Work into Agile Practices to Running at Different Stages of Product Development. Frequently, start-ups guide their software development by agile practices [20]. Nevertheless, we found that although the professionals see the UX relevance, they face difficulties of fitting UX work into agile practices, as shown in Table 5. Furthermore, our results showed that UX’s concerns are spread throughout the different frequency of application for start-ups that applied UX for up to 1 year (see details in Sect. 4.1). These results can indicate these start-ups are struggling to add the UX work to their software development activities. However, providing a good user experience (UX) from the beginning of product development can have a meaningful positive impact on product acceptance [5].

Challenge 2 - Making Practices Leaner for UX Work. When adopting UX practices, start-ups are often concerned with the cost their application can bring, requiring them to find a balance between the UX work application and the less resource-consuming [22]. The pressure to put the product on the market and the lack of UX professionals can be among the causes for a low frequency on the application of UX practices. A report on design maturity points that organizations with less expertise in UX have difficulty accommodating practices throughout the software development process [10]. From a similar perspective, the report results can explain why start-ups that have been applying UX for less than 1 year do not frequently use UX practices (see Table 3). Especially for early-stage start-ups, UX practices need to be adopted to do just enough to validate the product ideas without waste resources [5]. Nonetheless, a key question continues unanswered: “how to make UX practices more lightweight in order to be incorporated into the start-ups daily work?”.

Challenge 3 - Adjusting the Pace of UX Work in a Highly Reactive Environment. Our findings pointed out that creating successful products is a trigger for UX’s work (see Table 4). However, it may be harder for start-ups to add UX practices into their software development process by being highly reactive to market demands. When the start-up decides to change its product drastically, UX work may need to be started from scratch, e.g., conducting research with a new group of users [5]. In addition, UX professionals may have to deal with the frustration of discontinuing a project if they are not prepared to work at the pace of these innovation-driven environments. Start-ups do not always have UX specialists who may be involved in assessing the business model from the early stages.

Challenge 4 - Aligning UX Work with the Business Model and User Needs. In our survey, the most cited reasons for applying UX in start-ups are create value for the user and create value for business development, as shown in Table 4. However, the lack of resources and time to conduct UX activities appeared as factors that harm UX activities (see Table 5), while the understanding how to sustain the long-term business model is not seen as an important factor by professionals in start-ups that have UX-related roles (i.e., UX-Pro, see Table 4). This result could be a consequence of UX not being seen as a crosscutting attribute. Our results have similarities with others from the literature which also point out that UX impacts on the business model and is important to create a user-product linked [9, 12, 22].

Challenge 5 - Training and Skills Development to Perform UX Activities. Our results showed that professionals from start-ups that do not have a designated UX professional recognize the importance of this professional for carrying out UX work (see Table 5). Also, our results show that both professionals from start-ups with and without UX-related roles have similar perspectives about the skills needed to perform UX activities (see Table 6). In addition to expertise in UX, Hokkanen et al. [9] highlighted the expertise of domain as an associated factor that should drive how and what type of UX can be created and tailored to the start-ups’ business niche. In the same work, the participants reported that two key factors to UX work are the development of the abilities to get user feedback and to promote user involvement. Our results restate the concerns on collecting and handling user feedback and information (Table 6), as discussed in [5, 7, 9, 13]. However, we add a new detail by showing that these concerns are seen as important skills by professionals independently if they have or not UX-related roles. These findings lead us to the idea that there is a common consciousness about what skills should be developed or improved by start-ups professionals. We have not found out in the literature the best practices or lessons learned about how to introduce these skills in start-ups’ teams.

Challenge 6 - Conducting Research with Real Users. Although usability testing appeared as an often approach used by start-ups, the literature shows that professionals face difficulties in conducting user testing with real end-users [7]. In many situations, user testing is carried out with friends and with other internal members of the start-up. Differently of the literature, know-how to get information from users did not appear in the top as an obstacle to applying UX (Table 6). Hokkanen et al. [8], however, point that start-ups had challenges in collecting meaningful information from users and need of a systematically way to handle user information. Our sample did not provide us inputs to get an explanation of our different result. However, we consider this issue deserves a better further investigation.

6 Conclusions

This paper presented an investigation on the UX state-of-practice in software start-ups from a survey conducted with 88 professionals who work in software start-ups in Brazil. The respondents were in UX and SE positions. In our analysis, we addressed the role of UX in software development and the main practices used in the fast-paced environment of start-ups. We also looked at the reasons that influence or hinder UX work, the essential UX-related skills, and UX relevance from professionals’ point of view. In summary, our findings showed that even considering UX relevant, professionals have faced problems to fit UX in the software development work, regardless of having UX-related positions allocated. We concluded that more than developing professionals’ mindset about UX relevance, software start-ups need a set of practices to support decision-making about how and when UX work can be effectively embedded into their software development activities.

By analyzing these issues in a critical way, we have identified a set of challenges faced by software start-ups in the adoption of UX practices. The identification of these challenges is a significant contribution achieved by this work since these can be used to drive actions on start-ups and to guide future studies on the field. Besides, our discussion provides insights to encourage researchers and software practitioners to carry out more in-depth investigations on topics, such as the UX value for users and business, and UX in the volatile scenario of early-stage start-ups. Future work includes examining the remaining survey questions and the replication of our survey to other start-up ecosystems.