ORBi UMONS: Detailled Reference

Article (Scientific journals)

How healthy are software engineering conferences?

Vasilescu, Bogdan; Serebrenik, Alexander; Mens, Tom et al.

2014 • In Science of Computer Programming, Volume 89, Part C, p. 251-272

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/20.500.12907/12729

DOI
10.1016/j.scico.2014.01.016

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

scico2014-conferencehealth.pdf

Publisher postprint (714.34 kB)

All documents in ORBi UMONS are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

[en] empirical research; [en] scientometrics; [en] conferences; [en] software engineering

Abstract :

[en] In this article we study the health of software engineering conferences by means of a suite of metrics created for this purpose. The metrics measure stability of the community, openness to new authors, introversion, representativeness of the PC with respect to the authors' community, availability of PC candidates, and scientific prestige. Using this metrics suite, we assess the health of 11 software engineering conferences over a period of more than 10 years. In general, our findings suggest that software engineering conferences are healthy, but we observe important differences between conferences with a wide scope and those with a more narrow scope. We also find that depending on the chosen health metric, some conferences perform better than others. This knowledge may be used by prospective authors to decide in which conferences to publish, and by conference steering committees or PC chairs to assess their selection process.

Research center :

CREMMI - Modélisation mathématique et informatique

Disciplines :

Electrical & electronics engineering

Author, co-author :

Vasilescu, Bogdan

Serebrenik, Alexander ; Université de Mons > Faculté des Sciences > Génie Logiciel

Mens, Tom ; Université de Mons > Faculté des Sciences > Génie Logiciel

van den Brand, Mark G. J.

Pek, Ekaratina

Language :

English

Title :

How healthy are software engineering conferences?

Publication date :

31 January 2014

Journal title :

Science of Computer Programming

ISSN :

0167-6423

Publisher :

Elsevier, Netherlands

Volume :

Volume 89, Part C

Pages :

251-272

Peer reviewed :

Peer Reviewed verified by ORBi

Research unit :

S852 - Génie Logiciel

Research institute :

R300 - Institut de Recherche en Technologies de l'Information et Sciences de l'Informatique

Available on ORBi UMONS :

since 15 March 2014

Statistics

Number of views

1 (0 by UMONS)

Number of downloads

0 (0 by UMONS)

More statistics

Scopus citations^®

26

Scopus citations^®
without self-citations

25

OpenCitations

25

Bibliography

J. Chen, and J.A. Konstan Conference paper selectivity and impact Commun. ACM 53 6 2010 79 83
M. Franceschet The role of conference publications in CS Commun. ACM 53 12 2010 129 132
B. Meyer, C. Choppy, J. Staunstrup, and J. van Leeuwen Viewpoint - Research evaluation for computer science Commun. ACM 52 4 2009 31 34
D. Patterson, L. Snyder, and J. Ullman Best practices memo: evaluating computer scientists and engineers for promotion and tenure Comput. Res. News 11 4 August 1999 A-B
L. Fortnow Viewpoint - time for computer science to grow up Commun. ACM 52 8 2009 33 35
M.Y. Vardi Conferences vs. journals in computing research Commun. ACM 52 5 2009 5
J. Crowcroft, S. Keshav, and N. McKeown Viewpoint: Scaling the academic publication process to internet scale Commun. ACM 52 1 2009 27 30
K. Birman, and F.B. Schneider Viewpoint - program committee overload in systems Commun. ACM 52 5 2009 34 37
J. Freyne, L. Coyle, B. Smyth, and P. Cunningham Relative status of journal and conference publications in computer science Commun. ACM 53 11 2010 124 132
M. Franceschet The skewness of computer science Inf. Process. Manag. 47 1 2011 117 124
M. Eckmann, A. Rocha, and J. Wainer Relationship between high-quality journals and conferences in computer vision Scientometrics 90 2 2012 617 630
H.V. Jagadish The conference reviewing crisis and a proposed solution SIGMOD Rec. 37 3 2008 40 45
T. Systä, M. Harsu, and K. Koskimies Inbreeding in software engineering conferences http://www.cs.tut.fi/∼tsysta/ accessed November 2013
B. Vasilescu, A. Serebrenik, and T. Mens A historical dataset of software engineering conferences Int'l Conf. Mining Software Repositories (MSR) 2013 IEEE 373 376
The DBLP computer science bibliography, http://dblp.uni-trier.de, accessed October 2012.
G. Robles, and J. González-Barahona Developer identification methods for integrated data from various sources MSR 2005 ACM 1 5
C. Bird, A. Gourley, P. Devanbu, M. Gertz, and A. Swaminathan Mining email social networks MSR 2006 ACM 137 143
W. Poncin, A. Serebrenik, and M.G.J. van den Brand Process mining software repositories CSMR 2011 IEEE Computer Society 5 14
E. Kouters, B. Vasilescu, A. Serebrenik, and M.G. van den Brand Who's who in GNOME: using LSA to merge software repository identities Int'l Conference Software Maintenance - ERA Track 2012 IEEE 592 595
M. Goeminne, and T. Mens A comparison of identity merge algorithms for software repositories Sci. Comput. Program. 78 8 2013 971 982
B. Vasilescu, A. Serebrenik, M. Goeminne, and T. Mens On the variation and specialisation of workload - a case study of the Gnome ecosystem community Empir. Softw. Eng. 2013 10.1007/s10664-013-9244-1
B. Vasilescu, A. Serebrenik, P.T. Devanbu, and V. Filkov How social Q&A sites are changing knowledge sharing in open source software communities CSCW 2014 342 354 10.1145/2531602.2531659
Simple h-index estimation http://shine.icomp.ufam.edu.br/index.php accessed November 2013
J.E. Hirsch An index to quantify an individual's scientific research output Proc. Natl. Acad. Sci. USA 102 46 2005 16569 16572 (Pubitemid 41688817)
A. Capiluppi, A. Serebrenik, and A. Youssef Developing an h-index for OSS developers Int'l Conf. Mining Software Repositories (MSR) 2012 251 254
R Development Core Team R: A Language and Environment for Statistical Computing 2010 R Foundation for Statistical Computing Vienna, Austria
W. Javed, B. McDonnel, and N. Elmqvist Graphical perception of multiple time series IEEE Trans. Vis. Comput. Graph. 16 6 2010 927 934
P. Cowpertwait, and A. Metcalfe Introductory Time Series with R 2009 Springer
M. Holander, and D.A. Wolfe Nonparametric Statistical Methods 1973 Wiley
F. Wilcoxon Individual comparisons by ranking methods Biom. Bull. 1 6 1945 80 83
O.J. Dunn Multiple comparisons among means J. Am. Stat. Assoc. 56 1961 52 64
D.J. Sheskin Handbook of Parametric and Nonparametric Statistical Procedures 4th edition 2007 Chapman & Hall
K.R. Gabriel Simultaneous test procedures - some theory of multiple comparisons Ann. Math. Stat. 40 1 1969 224 250
D.W. Zimmerman, and B.D. Zumbo Parametric alternatives to the Student t test under violation of normality and homogeneity of variance Percept. Mot. Skills 74 3(1) 1992 835 844
E. Brunner, and U. Munzel The nonparametric Behrens-Fisher problem: asymptotic theory and a small-sample approximation Biom. J. 42 1 2000 17 25
F. Konietschke, L.A. Hothorn, and E. Brunner Rank-based multiple test procedures and simultaneous confidence intervals Electron. J. Stat. 6 2012 738 759
T. Jaki, and L. Hothorn Statistical evaluation of toxicological assays: Dunnett or Williams test - take both Arch. Toxicol. 87 11 2013 1901 1910 10.1007/s00204-013-1065-x
Y. Dajsuren, M.G.J. van den Brand, A. Serebrenik, and S. Roubtsov Simulink models are also software: modularity assessment Int'l ACM Sigsoft Conf. Quality of Software Architectures, QoSA '13 2013 ACM 99 106
B. Vasilescu, V. Filkov, and A. Serebrenik StackOverflow and GitHub: Associations between software development and crowdsourced knowledge SocialCom/PASSAT 2013 IEEE 188 195
B.M. Brown, and T.P. Hettmansperger Kruskal-Wallis, Multiple comparisons and efron dice Aust. N. Z. J. Stat. 44 4 2002 427 438
G. Antoniol, M. Di Penta, and M. Harman Search-based techniques applied to optimization of project planning for a massive maintenance project Int'l Conf. Software Maintenance (ICSM) 2005 IEEE 240 249
F. Khomh, M. Di Penta, and Y.-G. Guéhéneuc An exploratory study of the impact of code smells on software change-proneness Working Conf. Reverse Engineering 2009 IEEE 75 84
F. Konietschke nparcomp. Reference Manual 2012
C. Bird, E.T. Barr, A. Nash, P.T. Devanbu, V. Filkov, and Z. Su Structure and dynamics of research collaboration in computer science SDM 2009 826 837
W.S. Martins, M.A. Gonçalves, A.H.F. Laender, and G.L. Pappa Learning to assess the quality of scientific conferences: a case study in computer science Joint Conf. Digital Libraries (JCDL) 2009 193 202
Y. Manolopoulos A statistic study for the ADBIS period 1994-2006 ADBIS Research Communications CEUR Workshop Proceedings vol. 215 2006 CEUR-WS.org
P. Laplante, J. Rockne, P. Montuschi, T. Baldwin, M. Hinchey, L. Shafer, J. Voas, and W. Wang Quality in conference publishing IEEE Trans. Prof. Commun. 52 2 2009 183 196
P. Küngas, S. Karus, S. Vakulenko, M. Dumas, C. Parra, and F. Casati Reverse-engineering conference rankings: what does it take to make a reputable conference? Scientometrics 2013 1 15
R. Snodgrass Single- versus double-blind reviewing: an analysis of the literature SIGMOD Rec. 35 3 2006 8 21 (Pubitemid 44547578)
S. Zionts MCDM - if not a roman numeral, then what? Interfaces 9 4 1979 94 101
E. Triantaphyllou Multi-Criteria Decision Making Methods: A Comparative Study Applied Optimization 2000 Springer
M. Biryukov, and C. Dong Analysis of computer science communities based on DBLP European Conf. Research and Advanced Technology for Digital Libraries (ECDL) 2010 228 235
E. Elmacioglu, and D. Lee Oracle, where shall I submit my papers? Commun. ACM 52 2 2009 115 118
Z. Zhuang, E. Elmacioglu, D. Lee, and C.L. Giles Measuring conference quality by mining program committee characteristics Joint Conf. Digital Libraries (JCDL) 2007 225 234 (Pubitemid 350157985)
S. Sakr, and M. Alomari A decade of database conferences: A look inside the program committees Scientometrics 91 1 2012 173 184
O. Inanc, and O. Tuncer The effect of academic inbreeding on scientific effectiveness Scientometrics 88 3 2011 885 898
J.P.C. Kleijnen, and W.J.H.V. Groenendaal Measuring the quality of publications: new methodology and case study Inf. Process. Manag. 36 4 2000 551 570 (Pubitemid 30594067)
M.A.M. Souto, M. Warpechowski, and J.P.M. Oliveira An ontological approach for the quality assessment of computer science conferences Advances in Conceptual Modeling - Foundations and Applications Lecture Notes in Computer Science vol. 4802 2007 Springer 202 212
W.S. Martins, M.A. Gonçalves, A.H.F. Laender, and N. Ziviani Assessing the quality of scientific conferences based on bibliographic citations Scientometrics 83 1 2010 133 155
R. Klamma, M.C. Pham, and Y. Cao You never walk alone: Recommending academic events based on social network analysis Int'l Conf. Complex Sciences 2009 657 670
R. da Silva, J.P.M. de Oliveira, J.V. de Lima, V. Moreira, Statistics for ranking program committees and editorial boards, CoRR abs/1002.1060.
M.C. Pham, R. Klamma, and M. Jarke Development of computer science disciplines: a social network analysis approach Soc. Netw. Anal. Min. 1 4 2011 321 340
S. Yan, and D. Lee Toward alternative measures for ranking venues: a case of database research community Joint Conf. Digital Libraries (JCDL) 2007 235 244 (Pubitemid 350157986)
C. Gini Measurement of inequality of incomes Econ. J. 31 1921 124 126
D.A. Patterson The health of research conferences and the dearth of big idea papers Commun. ACM 47 12 2004 23 24
M.D. Hill, J.-L. Gaudiot, M. Hall, J. Marks, P. Prinetto, and D. Baglio A wiki for discussing and promoting best practices in research Commun. ACM 49 9 2006 63 64 (Pubitemid 44371763)
J.C. Mogul, and T. Anderson Open issues in organizing computer systems conferences SIGCOMM Comput. Commun. Rev. 38 3 2008 93 102
J. Howison, K. Crowston, and A. Wiggins Validity issues in the use of social network analysis with digital trace data J. Assoc. Inf. Syst. 12 2011 article 2
B. Vasilescu, A. Capiluppi, and A. Serebrenik Gender, representation and online participation: A quantitative study of StackOverflow ASE International Conference on Social Informatics 2012 IEEE 332 338
B. Vasilescu, A. Capiluppi, and A. Serebrenik Gender, representation and online participation: A quantitative study Interact. Comput. 2013 10.1093/iwc/iwt047
E. Ouweneel, P.M. Le Blanc, W.B. Schaufeli, and C.I. van Wijhe Good morning, good day: A diary study on positive emotions, hope, and work engagement Hum. Relat. 65 9 2012 1129 1154
T. Mens, M. Claes, P. Grosjean, and A. Serebrenik Studying evolving software ecosystems based on ecological models T. Mens, A. Serebrenik, A. Cleve, Evolving Software Systems 2013 Springer 297 326

Similar publications

Contact ORBi UMONS