7

Exploration and Explanation in Computational Notebooks

 3 years ago
source link: https://dl.acm.org/doi/10.1145/3173574.3173606
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

ABSTRACT

3173574.3173606.key.jpg

Computational notebooks combine code, visualizations, and text in a single document. Researchers, data analysts, and even journalists are rapidly adopting this new medium. We present three studies of how they are using notebooks to document and share exploratory data analyses. In the first, we analyzed over 1 million computational notebooks on GitHub, finding that one in four had no explanatory text but consisted entirely of visualizations or code. In a second study, we examined over 200 academic computational notebooks, finding that although the vast majority described methods, only a minority discussed reasoning or results. In a third study, we interviewed 15 academic data analysts, finding that most considered computational notebooks personal, exploratory, and messy. Importantly, they typically used other media to share analyses. These studies demonstrate a tension between exploration and explanation in constructing and sharing computational notebooks. We conclude with opportunities to encourage explanation in computational media without hindering exploration.

References

  1. Eric Allman. 2012. Managing Technical Debt. Communications of the ACM, 55, 5 (May 2012), 50--55.
  2. Mike Bostock. 2017. A Better Way to Code. (Apr. 2017). Retrieved September 11, 2017 from https://medium.com/@mbostock/a-better-way-to-code2b1d2876a3a0
  3. Nanette Brown, Ipek Ozkaya, Raghvinder Sangwan, Carolyn Seaman, Kevin Sullivan, Nico Zazworka, Yuanfang Cai, Yuepu Guo, Rick Kazman, Miryung Kim, Philippe Kruchten, Erin Lim, Alan MacCormack, Robert Nord. 2010. Managing technical debt in softwarereliant systems. In Proceedings of the FSE/SDP workshop on Future of Software Engineering Research (FoSER'10). ACM Press, New York, NY, 47--52.
  4. Ward Cunningham. 1992. The WyCash portfolio management system. Addendum to the proceedings on Object-oriented Programming Systems, Languages, and Applications (OOPSLA'92). ACM Press, New York, NY, 29--30.
  5. Tao Ding and Patrick Schloss. 2014. Dynamics and associations of microbial community types across the human body. Nature, 509, 7500 (Apr. 2014), 357--360.
  6. Distil. 2017. Retrieved September 11, 2017 from https://distill.pub/
  7. Brian Dunbar. 2010. NASA - Shuttle Computers Navigate Record of Reliability. (June 2010). Retrieved September 15, 2017 from https://www.nasa.gov/mission_pages/shuttle/flyout/flyf eature_shuttlecomputers.html
  8. Martin Fowler. 1999. Refactoring: improving the design of existing code. Addison-Wesley Professional, Reading, MA.
  9. Nahum Gershon and Ward Page. 2001. What storytelling can do for information visualization. Commun. ACM 44, 8 (Aug. 2001), 31--37.
  10. Brian Granger, Chris Colbert, and Ian Rose. 2017. JupyterLab: The next generation jupyter frontend. JupyterCon 2017.
  11. Philip Guo and Margo Seltzer. 2012. Burrito: Wrapping your lab notebook in computational infrastructure. USENIX Workshop on the Theory and Practice of Provenance (TaPP '12).
  12. Richard Harper and Abigail Sellen. 1995. Collaborative tools and the practicalities of professional work at the international monetary fund. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '95). ACM Press, New York, NY, 122129.
  13. Thomas Herndon, Michael Ash, and Robert Pollin. 2014. Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Cambridge journal of economics, 38, 2 (Dec 2013), 257--279.
  14. Jessica Hullman and Nicholas Diakopoulos. 2011. Visualization rhetoric: Framing effects in narrative visualization. IEEE transactions on visualization and computer graphics, 17, 12 (Dec. 2011), 2231--2240.
  15. Jessica Hullman, Steven Drucker, Nathalie Riche, Bongshin Lee, Daniel Fisher, and Eytan Adar. 2013. A deeper understanding of sequence in narrative visualization. IEEE Transactions on visualization and computer graphics, 19, 12 (Dec. 2013), 2406--2415.
  16. Jupyter. A gallery of interesting Jupyter Notebooks. Retrieved September 11, 2017 from https://github.com/jupyter/jupyter/wiki/A-gallery-ofinteresting-Jupyter-Notebooks
  17. Sean Kandel, Andreas Paepcke, Joseph Hellerstein, and Jeffrey Heer. 2012. Enterprise data analysis and visualization: An interview study. IEEE Transactions on Visualization and Computer Graphics, 18, 12 (Dec. 2012), 2917--2926.
  18. Mary Beth Kery, Amber Horvath, and Brad Myers. 2017. Variolite: supporting exploratory programming by data scientists. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '17). ACM Press, New York, NY, 1265--1276.
  19. Alison Kidd. 1994. The marks are on the knowledge worker. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '94). ACM Press, New York, NY, 186--191.
  20. Donald Knuth. 1984. Literate programming. The Computer Journal, 27, 2 (Feb. 1984), 97--111.
  21. Robert Kosara and Jock Mackinlay. 2013. Storytelling: The next step for visualization. Computer, 46, 5 (May 2013), 44--50.
  22. Kateryna Kuksenok, Cecilia Aragon, James Fogarty, Charlotte P. Lee, and Gina Neff. 2017. Deliberate Individual Change Framework for Understanding Programming Practices in four Oceanography Groups. Journal of Computer Supported Cooperative Work, 26, 4--6 (May 2017), 1--29.
  23. Paul McBurney and Collin McMillan. 2014. Automatic documentation generation via source code summarization of method context. In Proceedings of the 22nd International Conference on Program Comprehension (ICPC '14). ACM Press, New York, NY, 279--290.
  24. Peter Medawar. 2008. Induction and Intuition in Scientific Thought. Routledge.
  25. Emerson Murphy-Hill, Chris Parnin, and Andrew Black. 2012. How we refactor, and how we know it. IEEE Transactions on Software Engineering, 38, 1 (Jan. 2012), pp.5--18.
  26. Nature. 2017. Announcement: Transparency Upgrade for Nature Journals. Nature, 534, 7645 (Mar 2017), 288.
  27. Roger Peng, 2011. Reproducible research in computational science. Science, 334, 6060 (Dec. 2011), 12261227.
  28. Fernando Perez and Brian Granger. 2015. Project Jupyter: Computational Narratives as the Engine of Collaborative Data Science. Retrieved September 11, 207 from http://blog.jupyter.org/2015/07/07/projectjupyter-computational-narratives-as-the-engine-ofcollaborative-data-science/
  29. Roman Radle. 2017. Codestrates: Literate Computing with Webstrates. In Proceedings of the ACM on User Interfaces and Systems Technology (UIST '17). ACM Press, New York, NY.
  30. Dan Russell, Mark Stefik, Peter Pirolli, and Stuart Card. 1993. The cost structure of sensemaking. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '93). ACM Press, New York, NY, 269--276.
  31. Arvind Satyanarayan and Jeffrey Heer. 2014. Authoring narrative visualizations with ellipsis. Computer Graphics Forum, 33, 3 (Jun. 2014), 361--370.
  32. Edward Segel and Jeffrey Heer. 2010. Narrative visualization: Telling stories with data. IEEE transactions on visualization and computer graphics, 16, 6 (Nov. 2010), 1139--1148.
  33. Slava Shirokov. 2015. GitHub + Jupyter Notebooks = <3. (May 2015). Retrieved September 15, 2017 from https://github.com/blog/1995-github-jupyternotebooks-3
  34. Giriprasad Sridhara, Emily Hill, Divya Muppaneni, Lori Pollock, K. Vijay-Shanker. 2010. Towards automatically generating summary comments for java methods. In Proceedings of the IEEE/ACM international conference on Automated software engineering (ASE '10). ACM Press, New York, NY, 43--52.
  35. Aurélien Tabard, Wendy Mackay, and Evelyn Eastmond. 2008. November. From individual to collaborative: the evolution of prism, a hybrid laboratory notebook. In Proceedings of the ACM conference on Computer Supported Cooperative Work (CSCW '08). ACM Press, New York, NY, 569--578.
  36. Michele Tufano, Fabio Palomba, Gabriele Bavota, Rocco Oliveto, Massimiliano Di Penta, Andrea De Lucia, Denys Poshyvanyk. 2015, May. When and why your code starts to smell bad. In Proceedings of the 37th International Conference on Software Engineering. IEEE Press, 403--414.
  37. John Tukey. 1977. Exploratory data analysis. Pearson.
  38. Greg Wilson, Jennifer Bryan, Karen Cranston, Justin Kitzes, Lex Nederbragt, and Tracy Teal. 2017. Good enough practices in scientific computing. PLoS computational biology, 13, 6 (Jun, 2017).

Supplemental Material

Index Terms

Comments


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK