2025-06-04 05:57:48
What happens when a blockchain game combines rewards, competition, and social tasks to re-engage its community? That is the premise behind My Neighbor Alice’s $500K airdrop, announced as part of its in-game birthday celebration. Starting June 3, the campaign is rolling out the largest airdrop in the game’s history, with 500,000 $ALICE tokens earmarked for players.
But beyond the giveaway, the mechanics of this campaign suggest a broader strategy: reactivating dormant users, expanding player networks, and experimenting with multi-phase incentives that blend gameplay and social media engagement.
My Neighbor Alice is a multiplayer builder game built on the Chromia blockchain, where players collect resources, craft NFTs, and customize virtual land in a shared environment called the Lummelunda Archipelago.
The game made headlines when it was named Binance’s Project of the Year and has since become one of the more recognizable names in the niche world of Web3 gaming. What sets it apart is the focus on on-chain ownership, community-driven gameplay, and the ability to craft tradable items that exist as NFTs.
Unlike traditional games, where assets are locked to the game environment, Web3 games like Alice allow users to own and trade in-game items independently. This model has attracted both gamers and speculators, though interest in the genre has been cyclical.
The airdrop is divided into four waves, each spaced roughly a month apart. The timeline begins on June 3 with the campaign’s first phase titled Chapter One: A New Adventure. Here is how the reward distribution works:
\
To earn points, users must complete a mix of daily tasks, including:
\
Each month, the leaderboard resets, and a new set of challenges is introduced, giving both newcomers and returning players a fresh opportunity to compete.
A leaderboard ranks players based on how many points they accumulate by completing tasks. These tasks vary in difficulty and impact. For example:
\
Sharing a post on Twitter might be worth 10 points
Completing a complex in-game quest could be 100 points
Referring new users might stack higher multipliers
\
A Reward Board is the interface where players see available tasks, track their points, and claim eligible rewards. By gamifying participation, Alice introduces a feedback loop that rewards consistent effort over time.
The timing of this campaign appears strategic. While interest in blockchain gaming has dipped from its 2021 peak, developers are turning to community-focused rewards to reactivate interest. In a statement posted on the official My Neighbor Alice website, the team notes that the airdrop aims to “reward long-time supporters and bring in new players through layered incentives.” This is not just about giving away tokens. It is about boosting retention and increasing daily active users through low-friction tasks that can be done without deep gameplay experience.
However, critics of Web3 gaming often raise a fair concern: Are players engaging for the game, or just for the reward? If the latter, what happens after the airdrop ends?
The success or failure of this campaign may offer a signal to other blockchain games. Here’s why:
\
In traditional games, reward systems are carefully balanced to maintain long-term engagement. In blockchain games, where users can extract real-world value, the risk of “farm and flee” is much higher. The design of the airdrop campaign with recurring resets, content creation incentives, and real in-game progression seems to acknowledge this risk and attempts to mitigate it.
Airdrops in crypto often feel like short-term marketing stunts. But the My Neighbor Alice campaign shows more strategic depth, especially with the integration of game nights, referral loops, and a dynamic leaderboard. Still, the true test will be whether players stay after the rewards are claimed. Long-term retention, not wallet engagement, will determine the campaign’s success.
I also think this model could serve as a blueprint for other blockchain games that struggle with converting speculation into stable user growth. The balance of fun, incentives, and community interaction will remain key.
The $500K $ALICE airdrop is not just a marketing event. It is a case study in how Web3 games are evolving their incentive structures. Whether it becomes a best practice or a cautionary tale depends entirely on what the players choose to do next.
Don’t forget to like and share the story!
\
2025-06-04 05:53:10
2. Methodology and 2.1. Research Questions
3. Results and Interpretation and 3.1. Type of Problems (RQ1)
4. Implications
4.1. Implications for the Copilot Users
4.2. Implications for the Copilot Team
4.3. Implications for Researchers
6. Related Work
6.1. Evaluating the Quality of Code Generated by Copilot
6.2. Copilot’s Impact on Practical Development and 6.3. Conclusive Summary
In this section, we report the results of three RQs and provide their interpretation. The results of Copilot usage problems are categorized into two levels: categories (e.g., Suggestion Content Issue) and types (e.g., LESS EFFICIENT SUGGESTION). Meanwhile, the results for causes and solutions are organized as types only (e.g., Network Connection Error). We also provide the mapping relationship of Copilot related problems to their causes and solutions. As mentioned in Section 2.4.2, only causes that were proven to lead to the problems and solutions that could resolve the problems were extracted and provided in the results. Therefore, not all problems have corresponding causes and solutions. It is worth noting that due to the rapid update of Copilot, some problems and feature requests raised by users have already been addressed in newer releases of Copilot. We identified these two scenarios separately as two types of solutions, i.e., Bug Fixed by Copilot and Feature Implemented by Copilot. However, due to the absence of Copilot version information in the dataset, we could not consider the version information thoroughly in this study. To help understand the taxonomies of problems, causes, and solutions of using Copilot, we provide examples with the “#” symbol, which indicates the “GitHub Issue ID”, “GitHub Discussion ID”, or “SO Post ID” in the dataset (Zhou et al., 2024).
Fig. 2 presents the taxonomy of the problems extracted from our dataset. We have identified a total of 1,355 problems related to Copilot usage. It can be observed that Operation Issue (57.5%) accounts for the majority of problems faced by Copilot users. Furthermore, there are a notable number of users who have countered Compatibility Issue (15.6%) when using Copilot in different environments, followed by users who have raised Feature Request (15.0%) based on their user experience and requirements. Additionally, smaller percentages were identified as Suggestion Content Issue (4.4%), User Experience Issue (4.3%), and Copyright and Policy Issue (3.3%).
\ 3.1.1. Operation Issue (57.5%)
\ Operation Issue refers to a category of obstacles encountered by users when attempting to utilize some of the fundamental functions of Copilot. This category of problems is divided into six types, which are elaborated below.
\ • FUNCTIONALITY FAILURE refers to the abnormality of code generation-related features provided by Copilot. Copilot offers various interactive features to better engage with users, such as “previous/next suggestion”, “viewing all suggestions”, and “configuration of shortcut keys to accept suggestions”. Users may encounter exceptions when using these features. For example, a user reported that “Copilot no longer suggesting on PyCharm” after a period of not using it (Discussion #11199).
\
\ • STARTUP ISSUE refers to errors or malfunctions encountered by users attempting to run Copilot. This issue results in a complete failure of Copilot to execute and is typically accompanied by error messages. Such problems may arise either during a user’s initial usage of Copilot or unexpectedly after several successful runs. For example, a user failed to activate Copilot after installing it on VSCode, and received an error message stating “Cannot find module” (Issue #383).
\ • AUTHENTICATION FAILURE refers to the issues related to user login and authentication difficulties when using Copilot. Copilot requires users to log in to their GitHub account before using the service. Only users with access permissions (including paid subscriptions, student identity verification, etc.) can use the code generation service of Copilot. During the authentication process, users may encounter AUTHENTICATION FAILURE, resulting in the inability to use Copilot. For example, a user was repeatedly prompted with the message “waiting for GitHub Authentication” in IDEA, and was unable to log in (SO #72505280).
\ • ACCESSING FAILURE refers to the situation where users fail to access Copilot’s server, which often involves errors related to server connections. A user may encounter an error message like “GitHub Copilot could not connect to server” (Discussion #11801).
\ • INSTALLATION ISSUE refers to the problems encountered by users during the installation process of Copilot, including installation errors, inability to find installation methods, and other related problems. For instance, a user failed to install Copilot on VSCode insiders, and the server log showed “Error while installing the extension” (Issue #189).
\ • VERSION CONTROL ISSUE refers to the problems that users encounter when adjusting the version of Copilot or its runtime environment (e.g., IDE), including the inability to upgrade the Copilot version or abnormal issues like continuing to prompt for upgrades even after upgrading. For example, a user reported that “copilot plugin fails to update” when using it in IntelliJ IDEA (Discussion #17298).
\ Interpretation: We identified Operation Issue at various stages of user interaction with Copilot. Users tend to report these problems and seek assistance, making Operation Issue the most prevalent category of problems related to Copilot. FUNCTIONALITY FAILURE (226), AUTHENTICATION FAILURE (198), and STARTUP ISSUE (193), are the top three types of such problems. We attribute the higher frequency of the first two types to the deficiencies in Copilot’s feature design and stability, which are also influenced by users’ environments in which Copilot operates. AUTHENTICATION FAILURE mainly stems from particular details encountered during the login process when users need to access Copilot with their GitHub accounts.
\ 3.1.2. Compatibility Issue (15.6%)
\ This category covers the problems that arise from mismatches between Copilot and its runtime environment. Copilot operates as a plugin in various IDEs and code editors (e.g., VSCode and IntelliJ IDEA), and the complexity of the environments Copilot operates on can result in an increased number of compatibility issues. These problems are further classified into three types, which are elaborated below.
\ • EDITOR/IDE COMPATIBILITY ISSUE refers to issues arising from mismatches between Copilot and its IDE or editor. These problems typically manifest as Copilot being unable to operate properly in a specific IDE or editor. For example, a user previously found that Copilot “does not work in Neovim” while writing a Python program, even though the Copilot status showed that “Copilot: Enabled and Online” (SO #72174839).
\ • PLUG-IN COMPATIBILITY ISSUE refers to a type of matching issue that arises when Copilot and other plugins are activated and working together at the same time. Such problems can cause partial or complete malfunctions of Copilot and other plugins. They are usually identified through troubleshooting methods such as disabling Copilot or other plugins. For instance, one user reported that “a Keyboard shortcut conflict with Emmet” prevented him from receiving code suggestions generated by Copilot (Issue #47).
\ • KEYBOARD COMPATIBILITY ISSUE refers to the situation when the functionality of Copilot can not be used in some uncommon keyboard layouts. For example, a user with a German keyboard layout could not use most of the code generation-related features of Copilot (Discussion #7094).
\ Interpretation: Compatibility Issue arises from the complex environments in which users utilize Copilot, as well as the compatibility robustness of Copilot itself. In the case of EDITOR/IDE COMPATIBILITY ISSUE (136), VSCode, the platform officially recommended for using Copilot, has garnered a higher number of reported issues about compatibility. We also identified many problems in other widely used IDEs, like Visual Studio, IntelliJ IDEA, and PyCharm. The appearance of PLUG-IN COMPATIBILITY ISSUE (70) is less predictable, which often arises when using Copilot with other code completion tools.
\ 3.1.3. Feature Request (15.0%)
\ Feature Request refers to the features that users request to add or improve based on their experience and actual needs when using Copilot. These feature requests not only help improve the user experience of Copilot but also contribute to the exploration of how AI code generation tools like Copilot can better interact with developers. This category is further divided into four types, as shown below.
\ • FUNCTION REQUEST refers to the requests for developing new functions in Copilot, which typically arise from users’ genuine needs and difficulties encountered while utilizing the tool. For example, a user requested that Copilot should be able to “look at the context of a project with multiple files”, rather than generate code suggestions based on the context in a single file (SO #73848372).
\ • INTEGRATION REQUEST refers to a type of request for Copilot to be available on certain platforms or to be integrated with other plugins. This is mainly due to the desire of some users to use Copilot in the environments they are familiar with. For instance, a user called for “Support for Intellij 2022.2 EAP family” (Discussion #17045). The requests for integration also reflect the popularity of Copilot among developers to some extent.
\ • UI REQUEST refers to requests made by users for changes to the User Interface (UI) of Copilot, such as modifying the appearance of the Copilot icon. These requests generally aim to improve the visual effects and user experience of Copilot. For example, a user requested the addition of a “status indicator” to provide information about the current working status of Copilot (Issue #163).
\ • PROFESSIONAL COPILOT VERSION refers to requests from some users for a professional version of Copilot. These users are developers from certain companies who hope to receive more professional and reliable code generation services in their actual work. They may have higher requirements for the reliability and security of Copilot’s code, as well as team certification and other aspects. For example, a user asked for “an on-prem version for companies to purchase”, so that they could deploy Copilot in a local environment (Discussion #38858).
\ Interpretation: For FUNCTION REQUEST (115), we observed that users expressed a desire for greater flexibility in configuring Copilot to align more closely with their development habits. Common requests include the ability to accept Copilot’s suggestions word by word and to specify where Copilot should automatically operate in terms of file types or code development scopes. More innovative demands involve the need for Copilot to provide suggestions according to the whole project, as well as features like code explanation and chat functionality which have been provided in GitHub Copilot Chat (Zhao, 2023). INTEGRATION REQUEST (72) reflects the wish of developers to use Copilot in their familiar environments. This places greater demands on the Copilot team, as we have identified a significant number of Compatibility Issues.
\ 3.1.4. Suggestion Content Issue (4.4%)
\ This category of problems refers to the issues related to the content of the code generated by Copilot. The generation of code suggestions is the core feature of AI code generation tools like Copilot, and the quality of the suggestions directly determines whether users will adopt them. Therefore, the content of the generated code is naturally an area of concern for users, researchers, and the Copilot team. These problems are further divided into seven specific situations, which are detailed below.
\ • LOW QUALITY SUGGESTION refers to situations where Copilot is unable to comprehend the context sufficiently to generate useful code. Such code suggestions may not have any syntactical errors, but due to their poor quality, they are unlikely to be adopted by users. For instance, Copilot once generated an empty method containing only a return statement without meeting the requirements specified in the user’s code (Discussion #6631).
\ • NONSENSICAL SUGGESTION refers to the code suggestions provided by Copilot that are completely irrelevant to the needs of users. Such suggestions are considered almost unusable and provide little heuristic assistance to the user. For example, a user once received an inaccessible fake URL generated by Copilot, which was of no help with his programming task (Discussion #14212).
\ • SUGGESTION WITH BUGS refers to the situation where Copilot is able to generate relevant code based on the context, but the suggested code contains some bugs. This can result in the program being able to run, but not as the developer intended, or in some cases, it may cause errors or crashes. For example, a user reported that Copilot suggested using “setState(!state)” instead of “setState(true)”, which caused a logical bug in his code (Issue #43).
\ • INCOMPREHENSIBLE SUGGESTION refers to the situation where Copilot provides potentially useful code suggestions, but due to the complexity of the code or user’s lack of experience, they found it challenging to comprehend the suggested code and need more time to verify its correctness. For example, a user complained that “My Github Copilot just autocompleted it for me, then I scoured the internet trying to find information pertaining to it but could not” (SO #73075410).
\ • SUGGESTION WITH INVALID SYNTAX refers to the situation where the suggestions generated by Copilot may contain syntax errors that prevent the program from running properly. For example, a user found that the code suggestions provided by Copilot “missed curly brackets, leading to erroneous code when accepting suggestion” (Discussion #38941).
\ • LESS EFFICIENT SUGGESTION refers to the code suggestions generated by Copilot that are functionally correct and meet the requirements of users, but may suffer from suboptimal execution efficiency or convoluted logic, potentially impacting the overall quality of the code. For example, when a user requested Copilot to “find the cup with the most water” and “find the maximum amount of water in any cup”, the suggested code performed adequately but was not optimized for efficiency (Issue #162).
\ • INSECURE SUGGESTION refers to the code suggestions generated by Copilot that introduce security vulnerabilities. For example, a user indicated that the code suggestion lacked accountability for the sizes being read (Discussion #6636).
\ Interpretation: The quality of code suggestions is a critical factor in determining the capability of Copilot for practical code development. We identified a relatively small number of Suggestion Content Issues, possibly indicating that users are less inclined to report problems related to suggested code compared to usage-related problems. Among these problems, LOW QUALITY SUGGESTION (25), NONSENSICAL SUGGESTION (12), and SUGGESTION WITH BUGS (8) are the three most frequently reported types, while INSECURE SUGGESTION (2) and LESS EFFICIENT SUGGESTION (2) are less prevalent. This result implies that the main concern for users could be whether Copilot can provide code suggestions that have significant referential value.
\ 3.1.5. User Experience Issue (4.3%)
\ This category covers user feedback on their experience of using Copilot. Compared with Operation Issue, Copilot generally runs and functions as intended, but the user experience is suboptimal. User Experience Issue can provide insights into areas where Copilot could be improved. User Experience Issue can be further classified into four types, which are detailed below.
\ • POOR FUNCTIONALITY EXPERIENCE refers to a type of user experience problem where the usage of Copilot’s code generation-related functionalities is unsatisfactory. When such problems arise, Copilot’s functionalities remain operational, which is different from FUNCTIONALITY FAILURE. However, users expressed dissatisfaction with their experience when using these functionalities. These problems can often hinder the coordination between users and Copilot, and even decrease the efficiency of development work. Such problems often highlight areas where Copilot could be further enhanced and may potentially motivate users to propose some Feature Requests. For example, a user felt that the code automatically generated and popped up by Copilot was quite noisy (Issue #97).
\ • POOR SUBSCRIPTION EXPERIENCE refers to the obstacles that users encounter during the process of subscribing to the services of Copilot. Copilot offers several subscription methods (e.g., student verification, paid subscription), leading to some inconvenience for users during this process. For example, one user felt lost and was unsure about what to do next after setting up a billing (Discussion #19119).
\ • POOR PERFORMANCE refers to performance issues that occur when Copilot is running, which directly impacts the user experience. These problems include high CPU usage, long response times, and overly frequent server access. For example, a user complained that Copilot took “around 1-2 minutes for it to show one suggestion” on VSCode, which was very slow (Discussion #19491).
\ • POOR AUTHENTICATION EXPERIENCE refers to the inconvenience that users encounter when authenticating their identities before using Copilot. While users successfully navigate the login procedure, they may experience a suboptimal user experience. This could be due to factors such as an unwieldy process flow or the absence of explicit instructions. For example, a user complained that Copilot frequently “prompt to enable GitHub Copilot on every VSCode launch” which can be a significant source of frustration (SO #70065121).
\ Interpretation: User Experience Issues provide valuable insights into the direction for improving Copilot. Among the POOR FUNCTIONALITY EXPERIENCE (24), the most commonly reported problems involve Copilot’s inline suggestions that cause disruptions to the coding process of users (5) and the inconvenience of not being able to accept certain portions of the suggested code (2). These concerns align with some of the demands mentioned by users in Feature Request, e.g., setting when Copilot can generate code and the length of suggested code.
\ 3.1.6. Copyright and Policy Issue (3.3%)
\ Copilot is trained on a large corpus of open source code and generates code suggestions based on the users’ code context. The way in which Copilot operates raises concerns regarding potential copyright and policy issues, as expressed by some users. These problems are divided into three types, as shown below.
\ • CODE COPYRIGHT ISSUE refers to the concerns raised by some code authors regarding the unauthorized use of their open-source code by Copilot for model training. GitHub is currently one of the most popular webbased code hosting platforms, and since the release of Copilot, there have been suspicions among some code authors that their code hosted on GitHub has been used for training without proper consideration of their license. For example, a user “started migration” of his projects on GitHub because he was worried about his code being used to train Copilot without permission (Issue #150).
\ • CODE TELEMETRY ISSUE refers to the concerns expressed by users regarding Copilot collecting their code to generate suggestions, which may potentially result in the leakage of confidential code. Some users may also simply be unwilling to have their own code, as well as the code generated by Copilot for them, collected for other purposes. For example, a user is concerned that using Copilot might “give away API key” in his code (SO #70559637).
\ • VIOLATION OF MARKETPLACE POLICY is a specific case where a user reported that Copilot was able to be published on the VSCode marketplace despite using proposed APIs, while other plugins were prohibited. The user suspected that this behavior may be in violation of the Marketplace Policy (Issue #3).
\ Interpretation: The emergence of Copyright and Policy Issue reveals the concerns of users about the way Copilot works. Copilot is trained on multi-language open-source code and also needs to collect users’ code context during its operation to generate suggestions. These two facts have led people to pay more attention to copyright and intellectual property problems when using Copilot, especially in inhouse development.
\
:::info Authors:
(1) Xiyu Zhou, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(2) Peng Liang (Corresponding Author), School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(3) Beiqi Zhang, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(4) Zengyang Li, School of Computer Science, Central China Normal University, Wuhan, China ([email protected]);
(5) Aakash Ahmad, School of Computing and Communications, Lancaster University Leipzig, Leipzig, Germany ([email protected]);
(6) Mojtaba Shahin, School of Computing Technologies, RMIT University, Melbourne, Australia ([email protected]);
(7) Muhammad Waseem, Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland ([email protected]).
:::
:::info This paper is available on arxiv under CC BY 4.0 DEED license.
:::
\
2025-06-04 05:53:04
2. Methodology and 2.1. Research Questions
3. Results and Interpretation and 3.1. Type of Problems (RQ1)
4. Implications
4.1. Implications for the Copilot Users
4.2. Implications for the Copilot Team
4.3. Implications for Researchers
6. Related Work
6.1. Evaluating the Quality of Code Generated by Copilot
6.2. Copilot’s Impact on Practical Development and 6.3. Conclusive Summary
To answer the three RQs in Section 2.1, we established a set of data items for data extraction, as presented in Table 1. Data items D1-D3 intend to extract the information of problems, underlying causes, and potential solutions from the filtered data to answer RQ1-RQ3, respectively. These three data items could be extracted from any part of a GitHub issue, discussion, or SO post, such as the title, the problem description, comments, and discussions.
\ 2.4.1. Pilot Data Extraction
\ The first and third author conducted a pilot data extraction on 20 randomly selected GitHub issues, 20 discussions, and 20 SO posts, and in case of any discrepancies, the second author was involved to reach a consensus. The results indicated that the three data items could be extracted from our dataset. Based on the observation, we established the following criteria for formal data extraction: (1) If the same problem was identified by multiple users, we recorded it only once. (2) If multiple problems were identified within the same GitHub issue, GitHub discussion, or SO post, we recorded each one separately. (3) For a problem that has multiple causes mentioned, we only recorded the cause confirmed by the reporter of the problem or the Copilot team as the root cause. (4) For a problem that has multiple solutions suggested, we only recorded the solutions that were confirmed by the reporter of the problem or the Copilot team to actually solve the problem.
\ 2.4.2. Formal Data Extraction
\ The first and third authors conducted the formal data extraction from the filtered dataset to extract the data items. Subsequently, they discussed and reached a consensus with the second author on inconsistencies to ensure that the data extraction process adhered to the predetermined criteria. Each extracted data item was reviewed multiple times by the three authors to ensure accuracy. The data extraction results were compiled and recorded in MS Excel (Zhou et al., 2024).
\ It is important to note that not all collected data include the cause and solution of a problem. Although we selected closed GitHub issues and answered GitHub discussions and SO posts during the data collection phase, the specifics of each piece of data vary significantly. Sometimes, the respondents to a Copilot related problem might offer a solution without a detailed analysis of that issue, preventing us from extracting the underlying causes. In other situations, although the cause of a problem was identified, the user did not describe the specific resolution process. For example, a user found that Copilot “cannot work correctly on VSCode remote server” and realized it was due to “the bad network”, but did not provide any detailed solutions (Discussion #14907). Additionally, even when some responses provided both causes and solutions, they might not be accepted or proven effective by the problem’s reporter or the Copilot team members. For example, a user asked for “a way to set up GitHub copilot in Google Colab”, but the user neither accepted nor replied to the three proposed answers (SO #72431032). Therefore, we cannot consider any of the three answers as an effective solution to his problem.
\
\
To answer the three RQs formulated in Section 2, we conducted data analysis by using the Open Coding and Constant Comparison methods, which are two widely employed techniques from Grounded Theory during qualitative data analysis (Stol et al., 2016). Open Coding is not confined by pre-existing theoretical frameworks; instead, it encourages researchers to generate codes based on the actual content within the data. These codes constitute descriptive summarizations of the data, aiming to capture the underlying themes. In Constant Comparison, researchers continuously compare the coded data, dynamically refining and adjusting the categories based on their similarities and differences.
\ The specific process of data analysis includes four steps: 1) The first author meticulously reviewed the collected data and then assigned descriptive codes that succinctly encapsulated the core themes. For instance, the issue in Discussion #10598 was coded as “Stopped Giving Inline Suggestions”, which was reported by a user who noticed that his previously functioning Copilot had suddenly stopped providing code suggestions in VSCode. 2) The first author compared different codes to identify patterns, commonalities, and distinctions among them. Through this iterative comparison process, similar codes were merged into higherlevel types and categories. For example, the code of Discussion #10598, along with other akin codes, formed into the type of FUNCTIONALITY FAILURE, which further belongs to the category of Operation Issue. Once uncertainties arose, the first author engaged in discussions with the second and third authors to achieve a consensus. It should be noted that, due to the nature of Constant Comparison, both the types and the categories underwent several rounds of refinement before reaching their final form. 4) The initial version of the analysis results was further verified by the second and third authors, and the negotiated agreement approach (Campbell et al., 2013) was employed to address the conflicts. The final results are presented in Section 3.
\
:::info Authors:
(1) Xiyu Zhou, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(2) Peng Liang (Corresponding Author), School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(3) Beiqi Zhang, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(4) Zengyang Li, School of Computer Science, Central China Normal University, Wuhan, China ([email protected]);
(5) Aakash Ahmad, School of Computing and Communications, Lancaster University Leipzig, Leipzig, Germany ([email protected]);
(6) Mojtaba Shahin, School of Computing Technologies, RMIT University, Melbourne, Australia ([email protected]);
(7) Muhammad Waseem, Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland ([email protected]).
:::
:::info This paper is available on arxiv under CC BY 4.0 DEED license.
:::
\
2025-06-04 05:52:55
2. Methodology and 2.1. Research Questions
3. Results and Interpretation and 3.1. Type of Problems (RQ1)
4. Implications
4.1. Implications for the Copilot Users
4.2. Implications for the Copilot Team
4.3. Implications for Researchers
6. Related Work
6.1. Evaluating the Quality of Code Generated by Copilot
6.2. Copilot’s Impact on Practical Development and 6.3. Conclusive Summary
The research goal of this study is to systematically identity the problems that developers reported when using GitHub Copilot in development, as well as their underlying causes and potential solutions. We formulated three RQs to be answered in this study, which are detailed in Section 2.1, and Fig. 1 provides the overview of the research process.
RQ1: What are the problems faced by users while using Copilot in software development practice?
\ Rationale: GitHub Copilot is one of the most popular AI-assisted coding tools and has been widely used software development with 1.3 million paid users till Feb of 2024 (Wilkinson, 2024), and consequently it is important to understand the specific challenges and problems users face while using this tool in software development practice.
\ RQ2: What are the underlying causes of these problems?
\ Rationale: Understanding the causes of the problems identified in RQ1 is essential to developing effective solutions to address them. By identifying these causes, the study can provide insights into how to improve the design and functionality of Copilot.
\ RQ3: What are the potential solutions to address these problems?
\ Rationale: Exploring the solutions of the problems identified in RQ1 and the causes identified in RQ2 is essential to improving the user experience in practical development using Copilot. By identifying these solutions, the study can gain insight into potential solutions that enhance the functionality and usability of Copilot.
\
We collected data from three sources: GitHub Issues[1], GitHub Discussions[2], and SO posts[3]. GitHub Issues is a commonly used feature on GitHub for tracking bugs, feature requests, and reporting other issues related to software development projects, which allows us to capture the specific problems that users have encountered when coding with Copilot. GitHub Discussions is a feature provided by GitHub for open-ended discussions among project contributors and community members, which also offers a central hub for project-related discussions and knowledge sharing. The topics at GitHub Discussions can vary from technical questions and suggestions to usage issues associated with Copilot. Stack Overflow is a popular technology community that provides a public Q&A platform that addresses a broad spectrum of topics related to programming, development, and technology, which also includes inquiries about using Copilot.
\ Considering that Copilot was announced and started its technical preview on June 29, 2021, we chose to collect the data that were created after that date. The data collection was conducted on June 18, 2023. To answer RQ3, i.e., the solutions for addressing Copilot problems, we chose to collect closed GitHub issues, as well as answered GitHub discussions and SO posts. Specifically, for GitHub issues, we used “Copilot” as the keyword to search closed Copilot-related issues globally in the entire GitHub, and a total of 4,057 issues were retrieved. We also employed “Copilot” as a keyword to search answered posts in SO, resulting in 679 retrieved posts. Note that we did not use the “Copilot” tag for retrieval because the keyword-based method allows us to obtain a more exhaustive dataset. Different from GitHub issues and SO posts, GitHub discussions are organized into specific subcategories, with “Copilot” included as a subcategory under the overarching “Product” category. Given the high relevance of these discussions to Copilot, we collected all the 925 answered discussions under the “Copilot” subcategory.
We conducted the data labelling on the collected data to filter out those that cannot be used for this study. The criteria for filtering is as follows: the issue, discussion, or post should contain specific information related to the use of GitHub Copilot.
\ 2.3.1. Pilot Data Labelling
\ To minimize personal bias in the formal labelling process, the first and third authors conducted a pilot data labelling. For GitHub issues and discussions, we randomly selected 100 and 25 from each, making up 2.5% of the total count. Due to the small quantity of SO posts, we randomly selected 35, which constitutes 5% of the total posts. Selecting a certain proportion of data from different platforms respectively is to verify whether the criteria of the two authors are consistent across various data sources. The inter-rater reliability between the two authors was measured by the Cohen’s Kappa coefficient (Cohen, 1960), resulting in values of 0.824, 0.834, and 0.806, which indicate a reasonable level of agreement between the two authors. For any discrepancies in the results, the two authors engaged in discussions with the second author to reach a consensus. The results of pilot data labelling were compiled and recorded in MS Excel (Zhou et al., 2024).
\ 2.3.2. Formal Data Labelling
\ The first and third authors then conducted the formal data labelling. During this process, we excluded a large amount of data not related to our research. For instance, “Copilot” may refer to other meanings in some situations, such as the “co-pilot” of an aircraft. Additionally, Copilot might be mentioned in a straightforward manner without additional information, like a post mentioned, “You can try using Copilot, which is amazing”. We also excluded such cases of data since they could not provide useful information about the usage of Copilot. During the labelling process, any result on which the two authors disagreed was subject to discussion with the second author until an agreement was reached. Ultimately, the two authors collected 476 GitHub issues, 706 GitHub discussions, and 142 SO posts. The data labelling results were compiled and recorded in MS Excel (Zhou et al., 2024).
\
:::info Authors:
(1) Xiyu Zhou, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(2) Peng Liang (Corresponding Author), School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(3) Beiqi Zhang, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(4) Zengyang Li, School of Computer Science, Central China Normal University, Wuhan, China ([email protected]);
(5) Aakash Ahmad, School of Computing and Communications, Lancaster University Leipzig, Leipzig, Germany ([email protected]);
(6) Mojtaba Shahin, School of Computing Technologies, RMIT University, Melbourne, Australia ([email protected]);
(7) Muhammad Waseem, Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland ([email protected]).
:::
:::info This paper is available on arxiv under CC BY 4.0 DEED license.
:::
\
2025-06-04 05:52:49
:::info Authors:
(1) Xiyu Zhou, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(2) Peng Liang (Corresponding Author), School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(3) Beiqi Zhang, School of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(4) Zengyang Li, School of Computer Science, Central China Normal University, Wuhan, China ([email protected]);
(5) Aakash Ahmad, School of Computing and Communications, Lancaster University Leipzig, Leipzig, Germany ([email protected]);
(6) Mojtaba Shahin, School of Computing Technologies, RMIT University, Melbourne, Australia ([email protected]);
(7) Muhammad Waseem, Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland ([email protected]).
:::
2. Methodology and 2.1. Research Questions
3. Results and Interpretation and 3.1. Type of Problems (RQ1)
4. Implications
4.1. Implications for the Copilot Users
4.2. Implications for the Copilot Team
4.3. Implications for Researchers
6. Related Work
6.1. Evaluating the Quality of Code Generated by Copilot
6.2. Copilot’s Impact on Practical Development and 6.3. Conclusive Summary
With the recent advancement of Artificial Intelligence (AI) and Large Language Models (LLMs), AIbased code generation tools become a practical solution for software development. GitHub Copilot, the AI pair programmer, utilizes machine learning models trained on a large corpus of code snippets to generate code suggestions using natural language processing. Despite its popularity in software development, there is limited empirical evidence on the actual experiences of practitioners who work with Copilot. To this end, we conducted an empirical study to understand the problems that practitioners face when using Copilot, as well as their underlying causes and potential solutions. We collected data from 476 GitHub issues, 706 GitHub discussions, and 142 Stack Overflow posts. Our results reveal that (1) Operation Issue and Compatibility Issue are the most common problems faced by Copilot users, (2) Copilot Internal Error, Network Connection Error, and Editor/IDE Compatibility Issue are identified as the most frequent causes, and (3) Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version are the predominant solutions. Based on the results, we discuss the potential areas of Copilot for enhancement, and provide the implications for the Copilot users, the Copilot team, and researchers.
In software development, developers strive to achieve automation and intelligence that most of the code can be generated automatically with minimal human coding effort. Several studies (e.g., Luan et al. (2019), Robillard et al. (2010)) and software products (e.g., Eclipse Code Recommenders (Eclipse, 2019)) have been dedicated to improving the efficiency of developers through the development of systems that can recommend and generate code. Large Language Models (LLMs) are a type of natural language processing technique based on deep learning that is capable of automatically learning the grammar, semantics, and pragmatics of language, and generating a wide variety of contents. Recently, with the rapid development of LLMs, AI code generation tools trained on large amounts of code snippets are increasingly in the spotlight (e.g., AI-augmented development in Gartner Trends 2024 (Gartner, 2023)), making it possible for programmers to automatically generate code with minimized human effort (Austin et al., 2021).
\ On June 29, 2021, GitHub and OpenAI jointly announced the launch of a new product named GitHub Copilot (GitHub, 2024c). This innovative tool is powered by OpenAI’s Codex, a large-scale neural network model that is trained on a massive dataset of source code and natural language text. The goal of GitHub Copilot is to provide advanced code autocompletion and generation capabilities to developers, effectively acting as an “AI pair programmer” that can assist with coding tasks in real-time. Copilot has been designed to work with a wide range of Integrated Development Environments (IDEs) and code editors, such as VSCode, Visual Studio, Neovim, and JetBrains (GitHub, 2024c). By collecting contextual information like function names and comments, Copilot is able to generate code snippets in a variety of programming languages (e.g., Python, C++, Java), which can improve developers’ productivity and help them complete coding tasks more efficiently (Imai, 2022).
\ Since its release, Copilot has gained significant attention within the developer community, and it had a total of 1.3 million paid users till Feb of 2024 (Wilkinson, 2024). Many studies identify the effectiveness and concerns about the potential impact on code security and intellectual property (Pearce et al., 2022) (Bird et al., 2023) (Jaworski and Piotrkowski, 2023). Some prior research investigated the quality of the code generated by Copilot (Yetistiren et al., 2022) (Nguyen and Nadi, 2022), while others examined its performance in practical software development (Imai, 2022) (Barke et al., 2023) (Peng et al., 2023).
\ However, there is currently a lack of systematic categorization of the problems arise during the practical use of Copilot from the perspective of developers, as well as the causes behind them and solutions for addressing them. To this end, we conducted a thorough analysis of the problems
\
\ faced by software developers when coding with GitHub Copilot, as well as their causes and solutions, by collecting data from GitHub Issues, GitHub Discussions, and Stack Overflow (SO) posts, which would help to understand the limitations of Copilot in practical settings.
\ Our findings show that: (1) Operation Issue and Compatibility Issue are the most common problems faced by developers, (2) Copilot Internal Error, Network Connection Error, and Editor/IDE Compatibility Issue are identified as the most frequent causes, and (3) Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version are the predominant solutions.
\ The contributions of this work are that:
\ • We provided a two-level taxonomy for the problems of using Copilot in the software development practice.
\ • We developed a one-level taxonomy for the causes of the problems and the solutions to address the problems.
\ • We drew a mapping from the identified problems to their causes and solutions.
\ • We proposed practical guidelines for Copilot users, the Copilot team, and other researchers.
\ The rest of this paper is structured as follows: Section 2 presents the Research Questions (RQs) and research process. Section 3 provides the results and their interpretation. Section 4 discusses the implications based on the research results. Section 5 clarifies the potential threats to the validity of this study. Section 6 reviews the related work. Finally, Section 7 concludes this work along with the future directions.
\
:::info This paper is available on arxiv under CC BY 4.0 DEED license.
:::
\
2025-06-04 05:52:08
Willemstad, Curaçao, June 3rd, 2025/GamingWire/--Odds are now live on Cloudbet for the BLAST Austin Major — with an explosive storyline already unfolding: FaZe Clan announced Oleksandr “s1mple” Kostyliev joins the roster on loan from Natus Vincere for the tournament.
The switch comes on the back of benching Helvijs “broky” Saukants. s1mple’s reputation is enough to bring huge hopes, yet the markets are cautious: 10.1 is the price to win the tournament for FaZe outright.
Outright winner odds of June 3:
Vitality is still the favorite to win, but the addition of s1mple makes FaZe the largest wild card on the table — a high-risk, high-reward bet for those who are looking for volatility.
Swiss format makes each round a risk
Austin Major is held between June 3 to 22 with 32 teams and a three-round Swiss format. Each round is played as best-of-one until teams get eliminated or qualified. With so little room for error, even solid lineups can get punished by one map — and underdogs can excel.
“This is a bettor's enigma,” says Tim "Dktruman" de Lange, CS commentator and Cloudbet esports analyst, “Favorites are susceptible to upsets, and long shots with a big day — such as FaZe — can reverse the script. Savvy players are closely watching the initial rounds because they understand each victory or defeat causes immediate line movement.”
No-limit betting on the match winner is available on Cloudbet. Round handicaps, correct score, overtime markets, and round total combos such as "Team 1 to win and more than X rounds" are also among Cloudbet’s available esports betting markets.
s1mple's influence: Upside vs uncertainty
It is s1mple’s first Major since his stint with Team Falcons for the Shanghai Major last November. While his raw skill is not in doubt, there is uncertainty pertaining to FaZe’s cohesion and practice time with him on the team, especially after his debut - and FaZe’s early exit - at IEM Dallas in May.
“Bettors are playing cautiously,” adds Dktruman, “FaZe is not bet for superiority at 10.1 — they’re bet for mayhem. If s1mple gets off to a fast start and the team clicks, we may look back on this number being the steal of the century. There is no time to warm up, however, in this best-of-one.”
Team Vitality arrives with the strongest form, having won six consecutive events and riding a 30-match win streak.
While Vitality leads the field, challengers are not in short supply. MOUZ, Team Spirit, and Team Falcons are among the top contenders capable of testing the French side’s dominance. Of those, MOUZ has the most direct history — though not the results. In 2025 alone, Vitality has defeated them in three separate finals.
Still, MOUZ brings a deep and well-rounded lineup. With former Vitality player Spinx now leading alongside torzsi, Jimpphat, and xertioN, the team has shown flashes of brilliance. Their most recent meeting at IEM Dallas pushed Vitality to the limit on multiple maps, underscoring their potential to turn the tide under the right conditions.
The modern meta for CS2 prefers aggressive play: explosive riflers, aggressive site rushes, and aggressive AWPers. CS2 maps have shorter time-to-kill and closer angles, so rounds are frequently won and lost within the initial 20 seconds.
“Players such as ZywOo, frozen, maden, and now s1mple are all well-placed to take advantage,” says Dktruman, “This extends to round total and first-kill props, where smart bettors try to take advantage of tendencies of the players in the initial rounds.”
Note: Odds are subject to change and reflect current market trends.
Founded in 2013, Cloudbet is the world’s longest-running crypto casino and sportsbook. Over the past decade, players worldwide have placed millions of bets using over 40 different cryptocurrencies.
In 2024, Cloudbet introduced the most generous welcome offer and loyalty program online, featuring stacked rewards and guaranteed daily cash drops for frequent bettors.
With a wide selection of slots, live casino games, and sports markets—ranging from esports to Premier League and NFL player props—Cloudbet is the leader in secure crypto betting. Users can visit Cloudbet.com; Instagram (@cloudbetofficial); Twitter/X (@Cloudbet).
Tim “Dktruman” de Lange is a Dutch CS:GO and Dota 2 commentator and betting analyst. Known for his sharp line reads and tournament insights, he contributes expert coverage for Cloudbet esports. Users can follow him on X @dktrumantv.
Irene
Cloudbet
:::tip This story was published as a press release by Gamingwire under HackerNoon’s Business Blogging Program.
:::
\
