Important Things !!!
1. Label your answers to correspond to the labeling in the problem set (e.g. sub-problem 2a should receive a response labeled 2a).
2. Only use the reading which I provide.
Superintelligence (5pts)
2a. What is superintelligence? Identify three potential paths to superintelligence that do not involve whole brain emulation (WBE). Explain how superintelligence could be more likely than not even if each path to superintelligence is unlikely to be taken.
2b. What is WBE? If WBE is achieved, is it likely to be achieved without advance warning? Say why or why not. What’s one reason for thinking that WBE may be an easier path to superintelligence than alternatives.
2c. Recall Chalmers’s dancing qualia argument for functionalism. Summarize this argument. Then say how the argument can be taken to suggest that taking a specific path to superintelligence is more likely to produce conscious superintelligent systems than taking another specific path. You can rely on background assumptions about these paths as long as you make them explicit.
2d. Conditional on superhuman intelligent systems being created by humans, should we expect superintelligent systems to have goals and cognitive architectures that resemble our own?
2e. What is an intelligence explosion? How do the prospects for superintelligence bear on the probability of an intelligence explosion? Could either exist without the other? If so, say how. If not, say why not.
The Singularity (5pts)
3a. What is a singularity in the core sense Chalmers uses? Why might the singularity be worth caring about even if it has a low probability of happening?
3b. What’s an argument that Chalmers gives for human-level AI? What are two objections that Bostrom would raise to this argument?
3c. What is AI++? In your own words, present Chalmers’s argument for AI++. What is a defeater in the context of this argument? Identify two potential defeaters; evaluate their relative likelihood of preventing AI++.
3d. Construct an argument that is analogous to Chalmers’s argument for AI++ but which concerns a self-amplifying phenomenon aside from intelligence.
3e. Present an objection to the argument in 3d. Say whether or not there is a corresponding objection to Chalmers’s argument. If there is, compare the plausibility of the two objections. If there is not, offer a hypothesis for why there is not.
The Simulation Argument (5pts)
4a. What is the simulation hypothesis? What is the simulation argument?
4b. Could the simulation argument succeed even if the simulation hypothesis is false? If so, how? If not, why not?
4c. One objection to the simulation hypothesis is that we should reject it because it is a skeptical hypothesis. Another objection to the simulation hypothesis is that we should reject it because it is unverifiable. Explain why each of these objections is misguided.
4d. Describe two conceivable developments that could make it more or less likely that we are living in a simulation. Say how these developments should affect our confidence in the simulation hypothesis.
4e. How is functionalism relevant to the simulation argument? Should the simulation argument increase our confidence in the possibility of conscious computers? Explain.
READING Link:
“All Possible Views About Humanity’s Future Are Wild” (Links to an external site.) (Karnofsky); podcast (Links to an external site.) interview of Karnofsky. https://www.cold-takes.com/all-possible-views-about-humanitys-future-are-wild/
AlphaFold 2 is here: what’s behind the structure prediction miracle
https://deepmind.com/blog/article/Competitive-programming-with-AlphaCode
https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules
https://www.theatlantic.com/science/archive/2017/04/octopuses-do-something-really-strange-to-their-genes/522024/
