Conference Rankings, Transparency, and Community

As researchers, we all need, sooner or later, to account for how conferences and journals are ranked by governments and funding institutions. The rank of a specific venue can have large consequences not only on the career of people, but also on how scholar research is carried on in the community. In the formal methods community, the topic has been even more under the highlights after the recent downgrade of FoSSaCS from rank A to B by the iCORE ranking.

ETAPS 2026 in Turin included an ask-me-anything session over this topic, with one panelist for each major ETAPS conference including Robbert Krebbers for ESOP, Barbara König for FoSSaCS, Maurice ter Beek for iFS, and Dirk Beyer for TACAS. This is a brief (and incomplete) account of what has been said.

The session opens with some initial statements by the panelists, which give the audience a varied but substantially uniform viewpoint. The malcontent among the community is evident, but rankings are here whether we like it or not, so we should not limit ourselves to lamenting the problem but propose solutions as well, Dirk said.

Maurice agrees and adds that we should try to influence the rankings as a community, which is not easy: he tried to apply for being part of the iCORE committee, and not only he did not receive any reply, but, he notes, the committee that downgraded FoSSaCS does not contain any member of our community.

Barbara, being part of FoSSaCS steering committee, is obviously unhappy about the recent downgrading decision and says they plan to obtain a re-rank at the earliest opportunity. But the reason of the downgrade must be understood before doing that. One ingredient is that FoSSaCS has been ranked both as a theory conference and as a software engineering conference, and the h-index of theory conferences is usually lower than a software engineering one.

Robbert agrees that the community should step in and push for initiatives such as DORA, which aim at improving how research is assessed.

Questions from the audience open after the initial statements from the panelists. Some people are worried about the consequences of the downgrade, which, however, is too early to know. People notice that CONCUR has been downgraded in the same way, so FoSSaCS is not alone.

Somebody asks how members of the community can help the conferences to improve their rankings. Barbara explains that one important aspect is that the top people in the field should publish at the conference, because that’s one of the metrics used by iCORE. Another issue is the choice of the topic. Do we want FoSSaCS to become theory-only? It may help, but it is not clear if it is possible.

A keyword that emerges from the discussion is ’transparency’. A huge problem appears to be the lack of transparency of these indexes. Nobody knows how iCORE works behind the curtains. Even worse, CS Rankings has no committee at all. The community, and this is a recurring theme at this point, should push for more transparency.

Strategy-wise, panelists express diverse opinions about a proposal that people have talked about since some time, namely merging the ETAPS conferences into a single large one. From a numeric perspective it may make sense, but not everybody is convinced that it would really be a positive change.

If we do not like these rankings, then, somebody asks how the panelists would design their own ranking. Transparency is again the first keyword. Next come the usage of public data, the involvement of the community, and, maybe, mechanisms able to give enough weight to smaller communities.

People then go back talking about iCORE and how the community could influence it. Somebody asks, since this appears to be a zero-sum game, whether it is moral to lobby iCORE to do our community’s best interests if that means damaging somebody else. As in other moral issues, there’s not a definitive answer to this question, neither in general nor from our panelists.

The session is approaching the end. Some last remarks from people in the audience include a suggestion to move on from the conference-based model used in computer science and move to journals, such as what SIGPLAN did. Then, of course, the problem would be transferred to journal rakings, which seem to behave less erratically, though.

Finally, in an application of a modern version of Godwin’s law, any proper discussion nowadays cannot conclude without mentioning AI. Is the explosion of AI conferences influencing the problem for better or for worse? Apparently not that much. For example, IJCAI has been downgraded by Chinese rankings, so the mass typical of nowadays AI conferences seem to not be a source of imbalances.

We are at the end of the session. Maybe the available time is never enough to dissect a complex topic like this, but if one keyword has to be chosen to summarise the discussion, I think it is community. Each with their own opinions, we can do a lot together. Acting as a community, not as an heterogeneous set of people, will be more important than ever in the future.

People leave, ready to start talking about proper science again, which is what our job should be all about if only these issues were not so important. See you at the next talk, see you at the next ETAPS.