Research discovery

Has someone answered that question you have not worked out how to ask yet?

Recommender systems for academics are hard and in particular, I suspect they are harder than normal because definitionally the content should be new and hard to relate to existing stuff. Indeed, finding connections is a publishable result in itself.

Complicated interaction with systems of peer review. Could a normal recommender system such as canopy be made to work for academics? Could this integrate with peer review in some useful way? Can we have services like Canopy, Pinterest or keen for scientific knowledge? How can we trade of recall and precision for the needs of academics?

Moreover the information environment is challenging. I am fond of Elizabeth Aceso’s summary::

assessing a work often requires the same skills/knowledge you were hoping to get from said work. You can’t identify a good book in a field until you’ve read several. But improving your starting place does save time, so I should talk about how to choose a starting place.

One difficulty is that this process is heavily adversarial. A lot of people want you to believe a particular thing, and a larger set don’t care what you believe as long as you find your truth via their amazon affiliate link […] The latter group fills me with anger and sadness; at least the people trying to convert you believe in something (maybe even the thing they’re trying to convince you of). The link farmers are just polluting the commons.

My paraphrase: knowledge discovery would likely be intrinsically difficult in a hypothetical beneficent world with great sharing mechanisms, but the economics of attention, advertising and weaponised media mean that we should be suspicious of the mechanisms that we can currently access.

In particular, Aceso makes me worry that my scattershot approach to link sharing is possibly detracting from the value of this blog to the wider world.

  • papr — “tinder for preprints”

    We all know the peer review system is hopelessly overmatched by the deludge of papers coming out. papr reviews use the wisdom of the crowd to quickly filter papers that are considered interesting and accurate. Add your quick judgements about papers to those of thousands of scientists around the world.

    You can use the app to keep track of interesting papers and share them with your friends. Spend 30 min quickly sorting through the latest literature and papr will keep track of the papers you want to come back to.

    With papr you can filter to only see papers that match areas that interest you, keywords that match your interest, or papers that others have rated as interesting or high quality. Make sure your literature review is productive and efficient.

    I appreciate the quality problem is important, but I am unconvinced by their topic keywords idea. Quality is only half the problem for me and the topic-filtering problem looks harder.

  • Robin Hanson, of course, has much to say on potentially better mechanism design for scientific discovery. I have qualms about his implied cash rewards system crowding out reputational awards; I think there is something to be said for that particular economy using non-cash currency; but yes why not try it out?

  • arxiv sanity

    Aims to prioritise the arxiv paper-publishing firehose so that you can discover papers of relevance to your own interests, at least if those interests are in machine learning.

    Arxiv Sanity Preserver

    Built by [@karpathy]( to accelerate research. Serving last 26179 papers from cs.[CV|CL|LG|AI|NE]/stat.ML

    Includes twitter-hype sorting, TF-IDF clustering, and other such basic but important baby steps towards web2.0 style information consumption.

    The servers are overloaded of late, possibly because of the unfavourable scaling of all the SVMs that it uses, or the continued growth of Arxiv, or epidemic addiction to intermittent variable rewards amongst machine learning reseachers. You could run your own installation — it is open source — but the download and processing requirements are prohibitive. Arxiv is big and fast.

  • groundai Aims to affect both discovery and publishing, by providing community peer review.

    The potential of Community Peer Review Commenting will provide a way for researchers to ask for feedback about their work, then incorporating this feedback into revisions and to generate new ideas.

    Making this feedback openly accessible to everyone can help increase the public’s understanding and trust of scientific work and increase transparency.

    Having community support and a dialogue atmosphere inspire ideas to flow and be explored freely through insightful questions. In dialogue, people think together.

    […] Preprints discussions usually happen on twitter and facebook, but these comments are not housed with the preprint. We believe having the opportunity to provide feedback that is stored directly with the preprint will increase transparency and collaboration at all stages of the scientific process. We hope to see the dialogue becomes a part of the scholarly record.

    t;dr This renders papers in a friendly format for public annotation and links to related ones and supporting data etc easily.

  • gitxiv (source)

    In recent years, a highly interesting pattern has emerged: Computer scientists release new research findings on arXiv and just days later, developers release an open-source implementation on GitHub. This pattern is immensely powerful. […]

    GitXiv is a space to share links to open computer science projects. Countless Github and arXiv links are floating around the web. It’s hard to keep track of these gems. GitXiv attempts to solve this problem by offering a collaboratively curated feed of projects. Each project is conveniently presented as arXiv + Github + Links + Discussion. Members can submit their findings and let the community rank and discuss it. A regular newsletter makes it easy to stay up-to-date on recent advancements. It’s free and open.

    In terms of things that I will actually use, this source-code requirement idea is good. However, the site itself is no longer maintained and has fallen into disrepair.

    Perhaps they are superceded by…

  • papers with code, which is similar.

    The mission of Papers With Code is to create a free and open resource with Machine Learning papers, code and evaluation tables.

    We believe this is best done together with the community and powered by automation.

    We’ve already automated the linking of code to papers, and we are now working on automating the extraction of evaluation metrics from papers.

  • trendingarxiv (source):

    Keep track of arXiv papers and the tweet mini-commentaries that your friends are discussing on Twitter.

    Because somehow some researchers have time for twitter and the opinions of such multitasking prodigies are probably worthy of note. That is sadly beyond my own modest capacities. Anyway, great hack, good luck.

  • Journal / Author Name Estimator identifies potential collaborators and journals by semantic similarity search on the abstract

  • Grant matchmaker suggest people with similar grants inside the USA’s NIH

  • The Syllabus:

    I wonder if this techno-editorial system works?

    Each week we publish curated syllabi featuring pieces that cut across text, video and audio. The curation runs either along thematic lines — e.g. technology, political economy, arts & culture — or by media type such as Best of Academic Papers, Podcasts, Videos. You can also build your own personalised syllabus centered around your interests.

    Our approach rests on a mix of algorithmic and human curation: each week, our algorithms detect tens of thousands of potential candidates — and not just in English. Our human editors, led by Evgeny Morozov, then select a few hundred worthy items.

    It is run by a slightly crazy sounding guy, Evgeny Morozov.

    The way in which Morozov collects and analyses information is secret, he says. He doesn’t want to expand on how he compares his taxonomies with the actual content of videos, podcasts, books and articles. "That’s where our cutting-edge innovation lies."

    … The categorisation and scoring of all information is an initial screening. Everything is then assessed by Morozov and his assistants, several times, ultimately resulting in a selection of the very best and most relevant information that appears during a week, sorted by theme.

    I do not believe this solves a problem I personally face, but perhaps it solves a useful problem generally? I am somewhat bemused by the type of scientific knowledge diffusion process that this implies. Is converging on a canonical list of this week’s hottest think pieces what we can and/or should be doing?

Reading groups and co-learning

The Journal Club is a web-based tool designed to help organize journal clubs, aka reading groups. A journal club is a group of people coming together at regular intervals, e.g., weekly, to critically discuss research papers. The Journal Club makes it easy to keep track of information about the club’s meeting time and place as well as the list of papers coming up for discussion, papers that have been discussed in previous meetings, and papers proposed by club members for future discussion.

Paper analysis/annotation

Academic reading workflow problem?.

Finding copies

unpaywall and oadoi seem to be indices of non-paywalled preprints of paywalled articles. oadoi is a website, unpaywall is a browser extension. Citeseer also.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.