hckrnws

Actually, aren't all browser extensions a security nightmare?

Or has something changed recently?

Yeah parts of the article would still be as valid if this was about regular extensions.

The main difference is that AI extension, by design, send the content of the pages you browse to a server.

A malicious "calculator" extension could also send all the content to a server, and extension users don't really have an idea of what each extension is actually doing.

So skip the "Malware posing as AI browser extension" section, it's same kind of security issues as a malware calculator extension.

The legitimate AI extension's problems are more interesting.

Article wastes a bit more time on other security issues you get from using AI LLM in general. Those apply whether you're using a browser extension or chat.openai.com directly.

The valid point that applies to narrowly AI browser extension are:

1) it could send sensitive data you wouldn't have sent otherwise. Most people would know what they're doing when they explicitly paste the stuff on chat.openai.com. But when it's now automated via the extension DOM scraping, it's a bit harder to realize how much you're giving away.

2) And the hidden text prompt injection. That's interesting as now your attacker could be the website you browse, if you have configured too many plugins (Zapier plugin giving access to your email)

These 2 parts of TFA are imo novel security issues that only exist with AI browser extension, and are interesting.

If a calculator extension is caught sending any data at all over the network they immediately would be suspicious, but evey AI extension has plausible deniability when making any requests, most can send all the webpage including form inputs and still have such deniability.

That's actually what I thought the title was until reading your comment, and I agreed vehemently.

No, because a typical safe-to-run browser extension is written in such a way that it can be examined to see what it does. AI-based tools can’t be analyzed based on their code, so the only way to make them safe is by limiting their capabilities. Any such capability limit is likely to be either too constraining, not constraining enough, or require as much planning ability as the AI itself.

When you talk about not being able to analyze these based on their code do you mean because today they're all just calling out to OpenAI or whoever?

The risks listed in the article itself mostly seem to fall under the same, non-AI-extension, core problem of "you're given them all your data." And that's a risk for non-AI-based extensions too, but if you look at the code of an AI one, it's gonna be obvious that it's shipping it off to a third party server, right? And once that happens... you can't un-close that door.

(The risks about copyright and such of content you generate by using AI tools are interesting and different, but I don't know that I'd call them security ones.)

The prompt injection one is pretty interesting, but still seems to fall under "traditional" plugin security issues: if you authorize a plugin to read everything on your screen, AND have full integration with your email, or whatever, then... that's a huge risk. The AI/injection part makes it triggerable by a third-party, which certainly raises the alarm level a lot, but also: bad idea, period, IMO.

>When you talk about not being able to analyze these based on their code do you mean because today they're all just calling out to OpenAI or whoever?

I think that the issue here is that AIs are probabilistic in nature, meaning that you can't fully predict their behavior in a particular situation just by reading the code. Instead in a tipical (non AI poweered) extension, the code is a precise description of what the extension will do in every possible situation.

> When you talk about not being able to analyze these based on their code do you mean because today they're all just calling out to OpenAI or whoever?

I mean that ML models are inherently inscrutable, it is extremely hard to determine how they operate internally, so no-one can identify any definite boundaries of what it will and will not output, or why. Hence prompt engineering, Bing's Sydney alternate personality, and weird hallucinated image artifacts.

Sure, if a user is calling OpenAI, they obviously can't understand the details of how it generates text. But neither can OpenAI! And if it produces something surprising, there's no way to fix it by directly modifying the model, the only way to do it is via ML techniques in the first place.

The problem is the permission system. Like apps, extensions have an all-or-nothing attitude to permissions. Browsers should allow the user to be more specific about permissions, and let extensions think the user gave more permissions than they actually did. E.g. if extension insists that they need "access to entire filesystem", the browser should make the extension believe they have access to the entire filesystem, but of course the entire thing is sandboxed and the user can restrict the access behind the scenes.

Without this feature, extensions will keep insisting they need access, and the user will eventually fall for it.

> Like apps, extensions have an all-or-nothing attitude to permissions

Browser extensions needs to declare their permissions. With Manifest V3 we’re seeing even more need to declare permissions.

Any extension cannot do anything not explicitly granted to it by the user upon installation.

The issue is those extensions can withhold valuable functionality needlessly.

If I download $usefulWikipediaCompanionExtension whose functionality only depends on access to *.wikipedia.org but whose manifest demands permission on all sites, I'd like to be able to tell my browser "if I'm not really on Wikipedia, only show the extension a blank page."

That's a lot more work than saying "No" to using the malware.

It's common for various counterparties, including software, to ask for much more information than they need and possibly be doing untrustworthy things with it while also providing legitimate value to the end user.

I've lied about my birthday while signing up for websites before. I've also made ad-hoc email addresses with forwarding to conceal my main email address. I've given fictitious phone numbers and I've used the names of fictional characters. I do this because I benefit from the service but I don't trust the provider to use my information responsibly.

Not a logical leap to go from there to feeding fake data to extensions when they request data that the user deems unnecessary for their functionality.

Yeah: while declaring permissions sounds cool and tries to fit into the narrative of helping protect end users who don't know how to manage anything themselves, at the end of the day it first requires an extremely opinionated central entity in charge of listings which takes a role in attempting to mediate the incentive incompatibilities (something which should raise serious ethical red flags and begs the question of conflicts of interest with respect to that player and the market that they get to fully control) but then still not only doesn't work to prevent users from getting abused, it will never work: "this app has requested access to your birthday" might be easy for end users, but (if this must be an API; but like, to the extent to which birthday is a bad example, this generalizes to every other thing that people currently must grant as "permissions") the only actually-correct solution is to always provide a concrete random date to every app by default and then allow the user to go out of their way--and this must not, under any circumstance, be something the app is allowed to prompt for or have any visibility into: this must be something the user has to initiate through external UI--to say "I grant this app access to my real birthday" (which, to the app, would have to look like the user merely changed the setting on their birthday to some other random date, as opposed to "the user finally gave us permission to see the same date that they can share to every other app").

> It's common for various counterparties, including software, to ask for much more information than they need

I believe if you ask for very wide permissions, at least when publihsing a browser-extension in the Google Chrome-store, you will have to justify why those are needed (from a user-facing POV), and your extension will be subject for additional review.

The same also applies when creating other Google-related apps which uses APIs which Google deems sensitive or restricted: You will have to justify their usage and be prepared for a review.

It's not bullet-proof, but it's more than nothing.

Yes but this extension needs to send the content of webpages you visit to APIs. You're gonna give it explicit permission to effectively do whatever the hell it wants.

shout out to the Arc browser, which has it's own browser sandbox and WYSIWYG tools to build JS snippets that run in your browser. I'm not affiliated with them in any way, but they're really changing the way I look at browsing online.

Does that come on a CD along with Intel Arc GPUs? :D

Already commented something similar in another thread:

Why is the security policy for extensions still not architected like other web permissions?

There has been a shift on mobile already from "take it or leave it"-style permissions on install towards more fine grained control not overidable by the app manifest.

I think Browser extensions should behave similarly. Especially when it comes to which origins an extensions is allowed to act on.

The user should be able to restrict this regardless of the manifest, even forced to do.

Extensions that need to act on all or an unknown set of origins should require a big and scary prompt after installation, regardless of what the user agrees to during installation.

I say this as a happy user of uBlock origin and React DevTools.

But for the common user the default should be to deny permissions and require user interaction.

you can make a warning as big and scary as you can, and people will just blindly hit accept/agree/ok. the look/design of the banner is not what will stop people from hitting ok, as at this point, i don't think anything will

While this is historically true, if the text is human readable - ‘may be able to read and transmit to a third party any data you input, including credit card numbers and passwords’ - is fairly likely to raise awareness. It’s not effect, but it’s better than nothing.

It’s worth contrasting clear communication such as the above to a EULA designed by scummy companies to not be read, browsers presumably have nothing to gain by exposing malicious plugins, so they’re a good candidate for the former.

If only we could get Mozilla executive to implement something actually useful instead of whatever meme tech they’ve lost their nut over this week, that’d be nice.

In isolation this is true, but for most people they just want the product the extension is offering - skipping past boring warnings is a means to an end. There is also the issue of warning fatigue when extension authors normalise asking for more permissions - more warnings leads to less engagement.

One way to avoid this would be to have an extension market which highlights alternative extensions and how they differ in permissions. But it would be hard to maintain those relationships, create a new oppportunity to game trust, push responsibility onto the market owners, etc. And ultimately, many interact with proprietary products without a direct competitor e.g. if FAANGs made them. So I can't see it happening.

Click 'agree' on the next 3 prompts within 15 seconds to see a monkey throwing an ice cream cone at King Charles

Mobile doesn't give you control over which origins it contacts.

Yes you are right, that came down to me after I hit the submit button. But consider my train of thought more an associative one.

I'd like an UI similar to the mobile one. I brought up the origin thing because for lots of extensions I would like that kind of UI for origin control. Origin control is part of WebExtension API, but it's during installation, which forces even well-meaning developers to request overly broad permissions for some kinds of extensions.

an extension developer can scope their extension to only run on certain URLs, and if that list changes then chrome will automatically disable it until the user re-authorizes for the new set of URLs.

so they're not a total security nightmare if they're only authorized to run on sites where you don't enter any private data. for example, looking through my extensions list, the py3redirect that autmatically redirects python2 documentation pages to python3 pages doesn't request access to anything other than python.org.

but otherwise, yeah, you're giving permission to execute arbitrary code on any website you visit, which is about as compromised as your browser can get.

>Actually, the current AI situation may be even more perilous than Jurassic Park. In that film, the misguided science that brought dinosaurs back to life was at least confined to a single island and controlled by a single corporation. In our current reality, the dinosaurs are loose, and anyone who wants to can play with one.

I'm really tired of reading stuff like this above. Seriously, AI is a disruptive tech and some people will oppose any change, but this is too much. All of the "security issues" mentioned in the article are true for browser extensions,and perhaps even software in general.

Then the author talks about "copyright mess" just before describing how it is pretty much resolved in their company (copilot banned).

The only real "problem with AI" is really a "problem with cloud" or more precisely "problem with people's lack of understanding of it". Average people should be interested in finding software alternatives that don't undermine their privacy.

For example look at AI image up scaling. Every single android app other than mine sends user's images to a server somewhere. Are those images retained? Are they scanned for whatever "legal purposes" the maker deems adequate? No one knows. No one cares. Well specifically in the entire world about 90 people seem to care.

Why 90 people? Because that's how many users my android app has 6 months after release. (the app does all processing locally, free version is ad supported, paid version can be used 100% offline).

While true, the main problem the ChatGPT era presents is the ability to do powerful things with weakly defined understand.

This is like handing out footgun coupons to all citizens who become "of age" and saying it's cool cause they were already legally allowed to buy footguns.

> Yes, large language models (LLMs) are not actually AI in that they are not actually intelligent, but we’re going to use the common nomenclature here.

I'm sorry for the off-topic comment, but why do I keep seeing this? What am I missing here – is it that some people define intelligence as >= human, or that LLM are not intelligence because they're *just* statistical models?

It's a way for the author to distinguish himself as one who is neither a purveyor of, nor fooled by, the magic, grift, and cringy sci-fi fantasizing that currently comprises the majority of AI discussion.

Currently, most mentions of AI, outside of a proper technical discussion, are coming from crypto-tier grifters and starry-eyed suckers. Even further, a lot of discussions from otherwise technical people are sci-fi-tier fearmongering about some ostensible Skynet, or something, it's not quite clear, but it's clearly quite cringe. The latter is one of the many calibers of ammunition being used by AI incumbents to dig regulatory moats for themselves.

Anyway, I understand why the author is distinguishing himself with his LLM...AI disclaimer, given the above.

In my field it's accepted (by some) that you write "AI" for your grant proposal and say "ML" when you talk to colleagues and want to be taken seriously.

It feels a bit wrong to me, because as you say it's arguably a grift, in this case on the taxpayer who funds science grants. More charitably it might just be the applicant admitting that they have no idea what they are doing, and the funding agency seeing this as a good chance to explore the unknown. Still, unless the field is AI research (mine isn't) it seems like funding agencies should giving money to people who understand their tools.

Most people outside of academia understand AI to include way more than just ML. People refer to the bots in video games as AI and they are probably a few hundred lines of straightforward code.

I don't think there is anything wrong with using the colloquial definition of the term when communicating with funding agencies/the public.

Would those topics that "outside academia understands AI to include" be covered in http://aima.cs.berkeley.edu ?

When you say "bots in video games as AI" that's covered in the book titled Artificial Intelligence: A Modern Approach, 4th US ed. :

    II Problem-solving 
        3 Solving Problems by Searching    ...  63 
        4 Search in Complex Environments   ... 110 
        5 Adversarial Search and Games     ... 146 
        6 Constraint Satisfaction Problems ... 180 

Sure, it may be a few hundred lines of code, but it's still something that a Berkley written AI textbook covers.

Spelled out more for that section:

    Chapter 5   Adversarial Search and Games ... 146

    5.1   Game Theory ... 146 
        5.1.1   Two-player zero-sum games ... 147 
    5.2   Optimal Decisions in Games ... 148 
        5.2.1   The minimax search algorithm ... 149 
        5.2.2   Optimal decisions in multiplayer games ... 151 
        5.2.3   Alpha--Beta Pruning ... 152 
        5.2.4   Move ordering ... 153 
    5.3   Heuristic Alpha--Beta Tree Search ... 156 
        5.3.1   Evaluation functions ... 156 
        5.3.2   Cutting off search ... 158 
        5.3.3   Forward pruning ... 159 
        5.3.4   Search versus lookup ... 160 
    5.4   Monte Carlo Tree Search ... 161 
    5.5   Stochastic Games ... 164 
        5.5.1   Evaluation functions for games of chance ... 166 
    5.6   Partially Observable Games ... 168 
        5.6.1   Kriegspiel: Partially observable chess ... 168 
        5.6.2   Card games ... 171 
    5.7   Limitations of Game Search Algorithms ... 173

I think I have an original edition of that book somewhere. Good Old Fashioned AI.

My assignments (different book) for Intro to AI class were:

Boolean algebra simplifier. Given a LISP expression - for example (AND A (OR C D)) write a function to return the variables needed to make the entire expression TRUE. Return NIL if the expression is a paradox such as (AND A (NOT A)). The expressions that we were to resolve had on the order of 100-200 operators and were deeply nested. I recall that I wrote a function as part of it that I called HAMLET-P that identified terms of the form (OR 2B (NOT 2B)) and rapidly simplified them to TRUE.

Not-brute-force job scheduler. The job-shop scheduling problem ( https://en.wikipedia.org/wiki/Job-shop_scheduling ) with in order processing of multiple tasks that had dependencies. Any worker could do any task but could only do one task at a time.

The third one I don't remember what it was. I know it was there since the class had four assignments... (digging... must have been something with Prolog)

The last assignment was written in any language (I did it in C++ having had enough of LISP and I had a good model for how to do it in my head in C++). A 19,19,5 game ( https://en.wikipedia.org/wiki/M,n,k-game ). Similar to go-maku or pente. This didn't have any constraints that go-maku has or captures that pente has. It was to use a two ply min-max tree with alpha beta pruning. It would beat me 7 out of 10 times. I could get a draw 2 out of 10 and win 1 out of 10. For fun I also learned ncurses and made it so that I could play the game with the arrow keys rather than as '10,9... oh crap, I meant 9,10'.

And I still consider all of those problems and homework assignments as "AI".

From the digging, I found a later year of the class that I took. They added a bit of neural nets in it, but other topics were still there.

By way of https://web.archive.org/web/19970214064228/http://www.cs.wis... to the professors's home page and classes taught - https://web.archive.org/web/19970224221107/http://www.cs.wis...

Professor Dryer taught a different section https://web.archive.org/web/19970508190550/http://www.cs.wis...

The domain of the AI research group at that time: https://web.archive.org/web/19970508113626/http://www.cs.wis...

I agree that using a colloquial definition is fine. And I don't mean to be too harsh on people who use buzzwords in their grant proposal: it's just sort of the sea you swim in.

But I only wish we could say that a few hundred lines of code was "AI": that would mean funding for a lot of desperately needed software infrastructure. Instead AI is taken as synonymous with ML, and more specifically deep neural networks, for the most part.

I think there’s nuance to be had here. Terms have been overloaded, and individuals aren’t necessarily acting in bad faith. ML can be considered to be a subset of AI.

That being said, ML is extremely boring to me, and I really do think a lot of the research is an enormous grift. Hop on the bandwagon, read a stats book, flagrantly plagiarize it, submit to CS journal that no statisticians read, publish and don’t perish, rinse, repeat.

It feels like society has spent billions of dollars on bad academics continuously reinventing applied statistics over and over again, but now with Big Data and a brand refresh! It’s like a whole generation of academics watched one too many terrible Hollywood remakes. It broke their brains, and now they’re only doing remakes too.

They ran out of statistics content to steal, so now the latest and greatest thing is plagiarizing classical AI works from the late 20th century and calling it “reinforcement learning.”

It’s all very frustrating. We could’ve funded a Manhattan project for fusion power, but instead thousands of our most brilliant people are wasting their time and humanity’s carbon budget to create the most powerful spambot ever.

I think you're entirely wrong about this. Using the term AI or artificial intelligence directly invokes several centuries of cultural baggage about golems, robots, Terminators, androids and cyborgs and Matrix-squid.

Saying "large language models" does not. Saying "giant correlation networks" does not. Not to be too Sapir-Whorfian, but the terminology we use influences our conversations: terrorists, guerillas, rebels, revolutionaries, freedom-fighters.

Should a nuclear power station rebrand itself to avoid being associated with Hiroshima? I really don't get what you are trying to say.

If you choose your words carelessly, you get unintended results.

Telling me about the AI in your HR system that hunts for the best candidates brings along the cultural context of stories about AI. Telling me about the rules engine that ranks incoming CVs does not.

"terrorists, guerillas, rebels, revolutionaries, freedom-fighters" are all the same group of people being referred to in different ways depending on how the speaker wants you to feel about them. Once you start using a particular word, you adopt the same viewpoint.

"AI" is too loaded with cultural contexts which will cause people to make mistakes.

I think its the "just" statistical models part.

If you pull up the TOC for an AI textbook, you'll find lots of things that aren't "intelligent". Machine learning is just a subset of it. I recall a professor in the AI department back in the 90s working on describing the shape of an object from a photograph (image to text) based on a number of tools (edge detection was one paper I recall).

Also in AI is writing a deductive first order logic solver is covered in there as are min-max trees and constraint satisfaction problems.

http://aima.cs.berkeley.edu

https://www.cs.ubc.ca/~poole/ci/contents.html (note chapter 4)

https://www.wiley.com/en-us/Mathematical+Methods+in+Artifici...

People are trying to put a box around "AI" to mean a particular thing - maybe they want AI to mean "artificial general intelligence" rather than all the things that are covered in the intro to AI class in college.

I ultimately believe that trying to use a term that has been very broad for decades to apply to only a small subset of the domain is going to end up being a fruitless Scotsman tilting at windmills.

... And you know what, I think it does a pretty good job at being intelligent. https://chat.openai.com/share/01d760b3-4171-4e28-a23b-0b6565...

Very clever people have located true intelligence in the gaps between what an machine can do and what a human can. Therefore, to show that you aren’t a starry-eyed rube you put a disclaimer that you aren’t really talking about intelligence, but something that just looks and acts like it.

True intelligence is, of course, definitionally the ability to do things like art or… err, wait, sorry, I haven’t checked recently, where have we put the goalposts nowadays?

I’m hesitant to even call this moving the goal posts. Intelligence has never been solidly defined even within humans (see: IQ debate; book smart vs street smart; idiot savants).

It’s unsurprising that creating machines that seem to do some stuff very intelligently and some other things not very intelligently at all is causing some discontent with regard to our language.

I see a whole lot more gnashing of teeth about goalposts moving than I do about people proposing actual solid goalposts.

So what’s your definition?

> I’m hesitant to even call this moving the goal posts. Intelligence has never been solidly defined even within humans (see: IQ debate; book smart vs street smart; idiot savants).

> It’s unsurprising that creating machines that seem to do some stuff very intelligently and some other things not very intelligently at all is causing some discontent with regard to our language.

I think I agree about the language.

I don’t have a definition of intelligence. I don’t work in one of those fields that would need to define it, so my first attempt probably wouldn’t be very good, but I’d say intelligence isn’t a single thing, but a label we’ve arbitrarily applied to a bunch of behaviors that are loosely related at best. So, trying to say this thing is intelligent, this thing is not, is basically hopeless, especially when things that we don’t believe are intelligent are being made to exhibit those behaviors, one behavior at a time.

> I see a whole lot more gnashing of teeth about goalposts moving than I do about people proposing actual solid goalposts.

I might not see a ton of explicit “here are the goalpost” type statements. But, every time someone says “I’m using the term AI, but actually of course this isn’t intelligence,” the seem to me at least to be referencing some implicit goalposts. If there isn’t a way of classifying what is or isn’t intelligent, how can they say something isn’t it? I think the people making the distinction have the responsibility to tell us where they’ve made the cutoff.

Maybe I’m just quibbling. Now that I’ve written all that out, I’m beginning to wonder if I just don’t like the wording of the disclaimer. I’d probably be satisfied if instead of “this isn’t intelligence, but I’m going to call it AI,” people would say “Intelligence is too hard to define, so I’m going to call this AI, because why not?”

Conceptually Speaking you can reduce it down to Intelligence and strip out the Artificial Label.

So know the question is what is Intelligence. Our standardized testing Model tells us passing tests that Humans cannot would be considered intelligent.

Then add back in artificial to complete the equation.

Commercially the Term Ai Means nothing thanks to years of Machine Learning being labeled such. It's arbitrary and relays more to Group Think to avoid approaching that Intelligence is a Scalar Value and not a Binary Construct.

>So what’s your definition?

I say we take the word intelligence and throw it out the window. It's a bit like talking about the either before we discovered more about physics. We chose a word with an ethereal definition that may or may not apply depending on the context.

So what do we do instead? We define sets of capability and context and devise tests around that. If it turns out a test actually sucked or was not expansive enough, we don't get rid of that particular test. Instead we make a new more advanced test with better coverage. Under this domain no human would pass all the tests either. We could each individual sub test with ratings like 'far below human capability', 'average human capability', 'far beyond human capabilities'. These tests could be everywhere from emotional understanding and comprehension, to reasoning and logical ability, and even include embodiment tests.

Of course even then I see a day where some embodied robot beats the vast majority of emotional, intellectual, and physical tests and some human supremacist still comes back with "iTs n0t InTeLLigeNt"

Heh, Computers will never be intelligent, we will just moving the bar until humans can no longer be classified as intelligent.

Stable Diffusion doesnt make art, it makes photos. We can deem them art.

Its denoising software.

Ooh, this is a rare one! A comment directly noting the similarities between AI art with photography, but insisting both aren't art. You're in very historical company: https://daily.jstor.org/when-photography-was-not-art/

>Photography couldn’t qualify as an art in its own right, the explanation went, because it lacked “something beyond mere mechanism at the bottom of it.”

That has nothing to do with the technology, that has everything to do with the quality.

Is it art if I take a picture with the cap on? No. Is it art if I take a picture of a tan colored wall? No.

Is it art if I set up something beautiful and take a picture. Its closer to art than the previous few examples.

If I write a prompt that says: "a green bedroom with art work on the walls", to be inspired, that still isnt trying to be art.

Basically, have higher standards.

There's long been a divide between what people call hard vs soft AI, or strong vs weak AI, or narrow vs general. The definitions are a bit fuzzy, but generally a hard AI or strong AI would be able to think for itself, develop strategies and skills, maybe have a sense of self. Soft AI in contrast is a mere tool where you put something in and get something out.

Now some people don't like using the term AI for soft/weak/narrow AI, because it's a fleeting definition, mostly applied to things that are novel and that we didn't think computers were able to do. Playing chess used to be considered AI, but a short time after AI beat the human chess world master it was no longer considered AI. If you buy a chess computer capable of beating Magnus Carlsen today that's considered a clever algorithm, no longer AI. You see the same thing playing out in real time right now with LLMs, where they go from AI to "just algorithms" in record time.

Because we don't have a real handle on what "intelligence" actually is, any use of the word without defining it is essentially just noise.

Yeah this is exactly it. It’s interesting seeing a precision-oriented discipline (engineering) running into the inherently very, very muddy world of semantics.

“What do you mean it’s not intelligent?! It passed Test X!”

“Yes and now that tells us Test X was not a good test for whatever it is we refer to as ‘intelligence’”

> LLM are not intelligence because they're just statistical models

This is exactly it for me.

Are you intelligent or just a bunch of cells? Given that I can query it for all sorts of information that I don’t know, I would consider LLMs to, at the very least, contain and present intelligence…artificially.

I can query Wikipedia or IMDB for all sorts of information I don't know. I wouldn't consider the search box of either site to be "intelligent", so I don't know "query it for all sorts of information" is a generally good rubric for intelligence.

And if your brain is mostly a statistical model of the world, with action probabilities based on what parts of it happen to be excited at the moment?

How do we know that the brain is a statistical model of the world? It sounds like explaining an unknown phenomenon using the technology du jour - just 10/20 years ago, the brain was a computer.

This touches on a dichotomy that has fascinated me for decades, from the very beginning of my interest in AI.

One side of the dichotomy asserts that "if it walks like a duck..." that is, if a computer appears to be intelligent to us, then it must be intelligent. This is basically the Turing Test crowd (even though Turing himself didn't approve of the Turing Test as an actual test of AI).

On the other side, you have people who assert that the human mind is really just a super-complicated version of "X", where "X" is whatever the cool new tech of the day is.

I have no conclusions to draw from this sort of thing, aside from highlighting that we don't know what intelligence or consciousness actually are. I'm just fascinated by it.

The general notion is called "lumpers" and "splitters".

From the perspective of software, the lumpers are pretty much always wrong except for when they get a lucky guess. Think of a pointy-haired boss who weaponizes his wishful thinking with a brutal dismissal of all implementation details and imposes ignorantly firm deadlines, or an architecture astronaut who writes and forces upon everyone cruel interfaces and classes that are thoroughly out of touch with reality.

As they say: "it's more easy to lump splits than split lumps". The people who insist the statistical models have emergent behavior, or even worse, equate them with human brains are "lumpers" who lack imagination and have no desire to truly understand and model these things. They naively seek out oversimplifications and falsely believe they're applying Occam's Razor, but they're actually just morons. "Splitters" are by their very definition always technically correct, but create complex distinctions that either represent much deeper knowledge than necessary, or hallucination. Either way, both types are needed, and of course, society values the lumpers far more for essentially playing the lottery with their reputations by telling people what they want to hear.

https://en.m.wikipedia.org/wiki/Lumpers_and_splitters

So conversely, is the brain magic? And if so, if we look at the evolutionary lineage of neural networks, at which point did it become so?

I wouldn't say the brain is magic, just that we still don't know what consciousness and intelligence is. Could the complex emergent behaviour we call intelligence emerge from a statistical model? Maybe. Can we gain more insights on what intelligence is by studying these models? Definitely. On the other hand — Are there limits to large language models' capabilities that we haven't reached yet?

I don’t think we know that. The point of my comment is to poke a bit at human exceptionalism. I think we’re going to see something that’s hard to deny is intelligent come out of a combination of a world model and an RL agent within the next decade. But I’m sure some will try to keep moving the goalposts.

The brain carries state and is self-modifying, which is something that can‘t be said about mere statistical models.

Its interesting to see what it thinks about some ideas, like I ask, what 5 companies are best at marketing. My goal here is to be hypercritical of the companies it says because they are masters at manipulation. GPT3.5 was awful and confused advertising and marketing. GPT4 was perfect (Apple, Nike, Coke, Amazon, P&G)

As much as chatgpt doesnt want to give you answers because the fuzziness, it has the ability to make judgements on things like "This is the best" or "This is the worst".

Ofc with bias.

Does it have the ability or is it just generating text similar to what it has seen before? The two things are very different.

In this examples, it likely took that those companies are often praised about their marketing in the same sentence marketing is mentioned.

LLMs don't repeat text its seen before, it links words/tokens/phrases that are related. Its prediction, but the prediction isnt just copypasting a previous webpage.

Have you use chatgpt yet? I wouldn't delay. Heck you are here on HN, you basically have a responsibility to test it.

I've used it extensively. GPT4 is great, but it is not intelligent. I think its really weird and also totally understandable that people think it is.

It’s something so new and foreign that I’m deeply unsurprised that some feel it’s intelligent.

I personally don’t care one way or the other, whether it is or isn’t. What I care about is whether it’s useful.

Eh, please comprehensively define intelligent... I have a feeling that this may explain a lot about your answer.

Well, one clear thing about GPT4 that isn't intelligent is that it doesn't learn in situ. Knowledge has to be added to it via an external process. The prompt does allow it to condition further output based on "new" information but that isn't learning. Another thing GPT4 has trouble with is generalizing knowledge. While it is certainly able to generalize to a degree (more or less it is able to apply patterns in the training data from one domain to other domains) if you ask it to generalize to things not well represented in the training data but nevertheless obvious from the conceptual underpinnings thereof it fails. I see this frequently with complicated functional/function level programming. GPT4 gets hopelessly confused when you ask it about non-trivial functions which return or manipulate other functions, even though conceptually there is nothing confusing about it and, in fact, if you ask it about functions as first class objects, it can answer with reasonable text.

Thus, GPT4 can appear to have knowledge in the sense of generating text indicating such, but fail to use that knowledge. This is the most compelling indication to me of limited or total lack of intelligence. I believe that the vast majority of GPT4's "capabilities" amount to memorization and permutation, not the formulation of accurate models of things.

> is it that some people define intelligence as >= human

I just want to say that this seems to be how many, if not most people define intelligence internally. If an LLM gets something wrong or doesn't know something, then it must be completely unintelligent. (as if humans never get anything wrong!)

Clearly the test isn’t >= as ChatGPT is already more coherent than large swaths of the population. The AI test for some is that its intelligence >>> human intelligence. Which is funny because by that point in time, their opinion will be more than worthless.

Like with humans, there are intelligent ways to be wrong and unintelligent ways to be wrong.

LLMs do a whole lot of “wrong in a way that indicates it is not ‘thinking’ the way an intelligent human would.”

What's concerning about this is we are evaluating AI on a basis that humans are not subject to. LLMs in their current form are built on the knowledge of the internet, while humans have both the internet and realtime feedback from their own lives in the physical world. If a human brain could be trained the same way as an LLM, might it also connect seemingly unconnected ideas in a way that would appear as non-thought? Maybe, maybe not. LLMs seem to be biased heavily towards making best effort guesses on things it doesn't know about, whilst humans are far more modest in doing so. I just don't know if we're really at a point where we can conclusively decide that something isn't thinking just because it doesn't appear to be thinking by the standards we place upon ourselves.

AI's a very soft term, and there's long been a technical vs "casual" split in what it means. Five or ten years ago you'd say your photo was retouched with AI dust removal, say, and we'd all know what that means. And that there was a big gulf between that and the sci-fi "AI" of Blade Runner or Her or Star Wars, etc.

The user interface to Chat GPT and similar tools, though, has made a lot of people think that gap is gone, and that instead of thinking they are using an AI tool in the technical sense, they now think they're talking to a full-fledged other being in the sci-fi sense; that that idea has now come true.

So a lot of people are careful to distinguish the one from the other in their writing.

It's statistical models all the way down.

That is not a very good reason to call an entity unintelligent. There are uncontroversial models of human intelligence that are Bayesian.

That's what I'm alluding to.

Ah, apologies, I read your comment as alluding to statistics as a reason to dismiss intelligence in machines

There are uncontroversial models of human intelligence that are Bayesian

But they're still models. Anyone claiming that Bayesian/statistical models have intelligence is confusing the map for the territory.

I say that large language models are not intelligent because of the way they fail to do things. In particular, they fail in such a way as to indicate they have no mental model of the things they parrot. If you give them a simple, but very unusual, coding problem, they will confidently give you an incorrect solution even though they seem to understand programming when dealing with things similar to their training data.

An intelligent thing should easily generalize in these situations but LLMs fail to. I use GPT4 every day and I frequently encounter this kind of thing.

Is there a definition of intelligence that rules out large language models, but that does not also rule out large portions of humanity? A lot of people would readily admit that they don't have programming aptitude and would probably end up just memorizing things. Do we say those people are not intelligent?

It seems to me that the perceived difference is mostly in being able to admit that you don't know something, rather than make up an answer -- but making up an answer is still something that humans do sometimes.

I have to admit this is a genuinely interesting question. Language models demonstrably do have some models of the world inside of them. And, I admit, what I say that they aren't intelligent, I mostly mean they are very stupid, rather than like a machine or algorithm. Artificial stupidity is progress.

Ok, so from your other comment, I think this is where our definition of intelligence is breaking down...

Biological agents have a consistent world model based on their capabilities because an inconsistent model would lead to lack of reproduction or death. We could call this environmental intelligence.

Meanwhile we have LLMs that have appear to have what I would consider 'micro' world models for some things, but not a large consistent world model. I'm guessing this is due to a few things, but for example not being culled for bad world models would be one, and another is they are only grounded in text and we've not really explored multi-modal grounding in models very far.

I guess what's going to be interesting is to see how multi-modal and embodied models do as they are trained in the environment and create a more consistent world model.

I believe that the best way to understand these large language models is that they have models of patterns of text. To the extent that patterns of text are congruent with patterns in the world, they appear to function well, but I think, in the end, they are statistical models of text, not of the world, and that substantially limits their capabilities.

I do think multi-modal models will be interesting, but text is a very special sort of thing. It is widely available, semantically rich, and informationally pretty dense. I'm not sure there is such a nice set of properties for other modes. Consider that we have already almost reached training data exhaustion with text and it is, by far, the most voluminous/dense training mode there is.

> is it that some people define intelligence as >= human

Just like some people define stupid as <= them. Aptitude is a multivariate spectra. It is already hard to come up with a cutoff on a single measure, way harder to do so for a bunch of different skills that for some reason happen to correlate in humans (and sometimes they diverge wildly as in the case of savant syndrome).

More like intelligence == human. ChatGPT is superhuman in many ways.

Only skimmed through the article, it seems -AI from the title would be an old story?

Also, that huge 4.7MB image in the head of the article...

Another good reason to use uBlock Origin and select the “Block media elements larger than x KB” option (x defaults to 50).

Edit: Wow! I just tried loading the page and see that the ridiculously large image still loads. That’s a particularly obnoxious website: the image’s HTTP header says that its Content-Length is 0 so it still gets downloaded by the browser.

SEO, who needs it!

I wonder when we’ll start seeing computer viruses that communicate with a remote LLM in order to get help circumventing barriers.

Alternatively, maybe anti-virus software can phone home to get on-the-fly advice.

> Alternatively, maybe anti-virus software can phone home to get on-the-fly advice.

Modern antivirus software already does this, more or less. It's usually called something like "cloud scanning."

Browser Extensions Are a Security Nightmare - I guess you can add AI in front to make it seem new.

Exactly - it blows my mind how normalized the permission Access your data for all websites is (I think it's Read and Change all your data on all websites for Chrome). I use only one or two extensions because of this. Why does a procrastination tool need such an insanely broad permission?

I wrote a Chrome extension[1] that reads no data but places a colored translucent div over the page. It requires that same "change all your data" permission.

My takeaway lesson is that the permissions model for extensions is confusing and nearly useless.

[1] https://chrome.google.com/webstore/detail/obscura/nhlkgnilpm...

How would you allow changing page contents with a narrow permission?

I also have a Chrome extension that needs access to page content on all pages, for the purpose of making text easier to read.

I could see distinguishing between extensions that in any way exfiltrate data from the pages you view, versus extensions that process the DOM and do something locally, but never send the data anywhere.

This requires a bit closer vetting than Google currently does, I think. To demonstrate that all processing happens locally, we encourage our users to load various websites with our extension toggled off, then go into airplane mode, and then turn our extension on. This doesn't strictly guarantee that we're not separately exfiltrating data (we aren't), but it does prove that our core process happens locally.

There are hundreds of thousands of extensions, and none of them make Google any money. Hard to see how they could justify any serious manual review.

Yeah, it could make sense for them to structure their extension framework so that developers could work with website data in a sandbox, if their use case allows for it. That would enable developers who don't need to send data to a server for processing to prove that the data never leaves the user's machine.

Do you suppose it's possible that accessing the DOM to add a div implicitly requires access to page data?

I can see how many applications might want to read the page, but in my case it's not necessary. My extension tries to add a <div> under the <body> element, regardless of what's going on in the page. If there's no <body>, my extension stops working but the browser keeps going.

In short, if there were separate "read" and "write" permissions, I would only need "write". For privacy-concerned people, that's a very important distinction.

It would be more complex than that given you can write arbitrary JavaScript that can read anything it likes and send it anywhere.

If it operates on more than one domain, it needs those permissions to function based on how the permissions system works. You can limit those yourself in the settings page for the extension, but everything else is basically workarounds applied to avoid that permission.

For example, a web clipper operates on multiple domains, but it can avoid it by using activetab permission instead and then offering optional permissions if it wants when you click on the clipper extension icon.

If you want something to be done automatically on multiple domains, this is not possible without that permission. Not unless you want to annoy users with prompts.

Just because an extension can do that, doesnt mean they are sending your info to a server.

No, but (1) you are trusting the extension to not do that, and (2) even if you vet the extension now, it could change in the future. Or am I mistaken? My understanding is that by default, extensions update automatically. If you accept these permissions initially, then you implicitly accept them for any future update. The alternative is keeping track of and updating every extension manually, re-vetting each one every time.

Exactly.

But I think at the moment it's easier to get someone to install an extension as long it mentions GPT or AI.

I have perment unstoppable hiccups that have occurred in the last week or so. Nothing I have tried has made them stop in fact I just hit up more every time I try to report record anything. I would like to just breathe without having hiccups and it's not even a choice for me I'm not even permitted to even attempt to stop this Behavior May hiccups are constant and unending. I have run out of ideas of who to pursue for help this is just Agony I can't even breathe without constant hiccup interruption I don't know how to make it stop and I'll do anything at this point.

https://pubmed.ncbi.nlm.nih.gov/3395000/

In case you're not joking

I do not understand how is it possible that Internet browsers do not currently have already built-in firewall that allows the user to control where the connection requests in the browser in general -the tab, the loaded web, the addon- are going to and from, and filter them.

But what is the "AI" ran entirely locally? https://pagevau.lt/

The "Download for Chrome" link on that page is broken. "404. That’s an error. The requested URL was not found on this server. That’s all we know."

The issue is not AI, nor browser extensions per se, the issue is the lackluster permission system that Chrome extensions have, it's pretty similar to what Android had 7 (?) years ago, which should not be acceptable in 2023.

Pretty much every single extension that isn't uBlock Origin is a security nightmare.

Even unblock is, only takes the repository owners login to be taken an update pushed.

No. There are many good, secure browser extensions.

Such as?

Privacy Badger, 1Password, HTTPS Everywhere, Dark Reader, to name a few.

> Add "Dark Reader"?

> It can: Read and change all data on all your websites

It already has the broadest permissions available. Dark Reader injects arbitary code into every page you visit. It's one silent update away from stealing all your sessions. This is a security nightmare.

All browser extensions are a security nightmare.

If you have the time, will, and ability, audit the latest release and turn off auto update. That’s counter productive when the extension has its own attack surface of course.

I also haven’t read anything concerning about Mozilla’s Recommended review system yet.

Interesting!

Automatic updates should be disabled by default...

[dead]