Gary Marcus在联合国大会谈AI风险：近期威胁是虚假信息与网络攻击，而非灭绝

Earlier today as part of the UNGA (United Nations General Assembly) Digital Cooperation Event, which also featured Yoshua Bengio and the Nobel Laureate Maria Ressa, I gave some “framing” remarks: We are at a moment of moral, political, and economic crisis. The choices that we make now around AI may reverberate for generations. Everybody in this room knows that. Although I think it is maybe still possible to achieve a net positive —or even great — outcome with AI, perhaps transforming technology and medicine, what I see in the discourse around AI frightens me. Extremes are how you get media. But none of the extremes makes sense. One extreme is absolutely zero regulation. Let AI rip, and hope for the best. Hope is not a strategy. Another extreme, increasingly popular, is terror: “doomers” insisting that literal extinction is imminent. But that outcome, taken literally, is extremely unlikely; every scenario for that that I have seen underestimates human ingenuity and human resourcefulness. (Many also overestimate AI.) What we are facing near-term, anyway, is not extinction. And it’s not superintelligence… It’s wholesale deepfaked disinformation, and unreliable but persistent AI systems stealing credentials and launching cyberattacks, at scale. (We may also face a recession, if the massive bets on AI are unsustainable.) We should expect that malicious actors will try use current AI to undermine financial systems and other vital infrastructure such as trains or electrical systems. (Recall the WannaCry virus took down UK’s hospital system for a few days.) What we actually need right now is increased reliability, better cybersecurity, and genuine enforcement, perhaps extending to product recalls and even prosecution of companies that repeatedly put dangerous products on the market. Our only hope is to take a sober look at where we are, embracing the practical over the sexy and the vague. Scientists, engineers and other people in civil society can do a lot here that isn’t sexy. But is necessary. The whole WannaCry fiasco could have been largely avoided if hospitals had kept up on cybersecurity procedures. So-called “rogue AI incidents” could largely be avoided if governments simply banned so-called AI agents with unrestricted internet access, until they could be shown to be safe. We need to hold AI companies accountable for the harms their products cause. And we need to replace vague talk about alignment with concrete steps like improving sandboxing and monitoring to better contain current systems. We also need to wean ourselves from an addiction to large language models, and to foster more research into outside-the-box alternatives that are more interpretable and more tractable. About the worst thing that we could do is to allow OpenAI to do what OpenAI has just done, which is to release a new model that is harder to monitor than its predecessors. Monitoring is an absolute foundation of cybersecurity, and sacrificing it is a mistake. They should not have been allowed to make that decision on their own. At a minimum, the three most important immediate steps I favor are these • The establishment and empowerment of an international advisory commission to evaluate the risks versus benefits of new systems before they are introduced. Independent scientists must have a voice. • An international advisory commission to audit new systems after they are released, on an ongoing basis. Again, independent scientists must have a voice, with real authority to demand change or even recall systems that prove dangerous. • An international agreement to not deploy architectures that are are demonstrably harmful. We must hold the frontier labs accountable for the lax security. I look forward to today’s discussion! Thank you very much. § You can find the whole session here: and here: § Now here is the wild and possibly wonderful plot twist. At almost literally the same time, more than the 20 countries signed a call for actions around AI — all apparently written in the last week — that captures a remarkable amount of what I have long hoped for: Many questions remain about how all this will be realized, financed and governed. But is a thrilling first step! Subscribe now Signatories thus far are listed below: