A Dataset of Bot and Human Activities in GitHub

[en] Software repositories hosted on GitHub frequently use development bots to automate repetitive, effort intensive and error-prone tasks. To understand and study how these bots are used, state-of-the-art bot identification tools have been developed to detect bots based on their comments in commits, issues and pull requests. Given that bots can be involved in many other activity types, there is a need to consider more activities that they are carrying out in the software repositories they are involved in. We therefore propose a curated dataset of such activities carried out by bots and humans involved in GitHub repositories. The dataset was constructed by identifying 24 high-level activity types that could be extracted from 15 lower-level event types that were queried from GitHub's event stream API for all considered bots and humans. The proposed dataset contains around 834K activities performed by 408 bots and 655 humans involved in GitHub repositories, during an observation period ranging from 25 November 2022 to 9 March 2023. By analysing the activity patterns of bots and humans, this dataset could lead to better bot identification tools and empirical studies on how bots play a role in collaborative software development.

Disciplines :

Computer science

Author, co-author :

Chidambaram, Natarajan ; Université de Mons - UMONS

Decan, Alexandre ; Université de Mons - UMONS > Faculté des Sciences > Service de Génie Logiciel

Mens, Tom ; Université de Mons - UMONS > Faculté des Sciences > Service de Génie Logiciel

Language :

English

Title :

A Dataset of Bot and Human Activities in GitHub

Publication date :

15 May 2023

Event name :

International Conference on Mining Software Repositories (MSR_

Event date :

15-16/5/2023

Audience :

International

Main work title :

Proceedings of the 20th International Conference on Mining Software Repositories (MSR 2023)

Publisher :

IEEE

Peer reviewed :

Peer reviewed

Additional URL :

https://ieeexplore.ieee.org/document/10173999

Research unit :

S852 - Génie Logiciel

Research institute :

Infortech
R300 - Institut de Recherche en Technologies de l'Information et Sciences de l'Informatique
R150 - Institut de Recherche sur les Systèmes Complexes

Funders :

DigitalWallonia4.AI research project ARIAC
F.R.S.-FNRS - Fonds de la Recherche Scientifique [BE]

Funding number :

2010235; F.4515.23; O.0157.18F-RG43; T.0149.22

Funding text :

This work is supported by DigitalWallonia4.AI research project ARIAC (grant number 2010235), as well as by the Fonds de la Recherche Scientifique – FNRS under grant numbers F.4515.23, O.0157.18F-RG43 and T.0149.22.

Data Set :

A Dataset of Bot and Human Activities in GitHub

Available on ORBi UMONS :

since 16 March 2023

Statistics

Number of views

27 (15 by UMONS)

Number of downloads

5 (5 by UMONS)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb, “Social coding in GitHub: Transparency and collaboration in an open software repository,” in International Conference on Computer Supported Cooperative Work (CSCW). ACM, 2012, pp. 1277–1286.
J. M. Costa, M. Cataldo, and C. R. de Souza, “The scale and evolution of coordination needs in large-scale distributed projects: implications for the future generation of collaborative tools,” in SIGCHI Conference on Human Factors in Computing Systems. ACM, 2011, pp. 3151–3160.
M. Wessel, B. M. De Souza, I. Steinmacher, I. S. Wiese, I. Polato, A. P. Chaves, and M. A. Gerosa, “The power of bots: Understanding bots in OSS projects,” International Conference on Human-Computer Interaction (CHI), 2018.
M. Golzadeh, T. Mens, A. Decan, E. Constantinou, and N. Chidambaram, “Recognizing bot activity in collaborative software development,” IEEE Software, vol. 39, no. 5, pp. 56–61, 2022.
N. Chidambaram, A. Decan, and M. Golzadeh, “Leveraging predictions from multiple repositories to improve bot detection,” in International Workshop on Bots in Software Engineering (BotSE). IEEE, 2022.
Z. Wang, Y. Wang, and D. Redmiles, “From specialized mechanics to project butlers: The usage of bots in open source software development,” IEEE Software, vol. 39, no. 5, pp. 38–43, 2022.
M. Golzadeh, A. Decan, D. Legay, and T. Mens, “A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments,” Journal of Systems and Software, vol. 175, 2021.
A. Abdellatif, M. Wessel, I. Steinmacher, M. A. Gerosa, and E. Shihab, “BotHunter: An approach to detect software bots in GitHub,” in International Conference on Mining Software Repositories (MSR), 2022, pp. 6–17.
J.-G. Young, A. Casari, K. McLaughlin, M. Z. Trujillo, L. Hébert-Dufresne, and J. P. Bagrow, “Which contributions count? Analysis of attribution in open source,” in International Conference on Mining Software Repositories (MSR), 2021, pp. 242–253.
S. Onoue, H. Hata, and K.-I. Matsumoto, “A study of the characteristics of developers’ activities in GitHub,” in Asia-Pacific Software Engineering Conference (APSEC), vol. 2. IEEE, 2013, pp. 7–12.
Y. Ma, C. Bogart, S. Amreen, R. Zaretzki, and A. Mockus, “World of code: an infrastructure for mining the universe of open source VCS data,” in International Conference on Mining Software Repositories (MSR). IEEE, 2019, pp. 143–154.
G. Gousios and D. Spinellis, “GHTorrent: Github’s data from a firehose,” in Working Conference on Mining Software Repositories (MSR). IEEE, 2012, pp. 12–21.
C. Bird, P. C. Rigby, E. T. Barr, D. J. Hamilton, D. M. German, and P. Devanbu, “The promises and perils of mining git,” in Working Conference on Mining Software Repositories (MSR). IEEE, 2009, pp. 1–10.
E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. German, and D. Damian, “An in-depth study of the promises and perils of mining GitHub,” Empirical Software Engineering, vol. 21, no. 5, pp. 2035–2071, 2016.