hckrnws

Looks fantastic but does it require using your endpoints for inference? We have strict data requirements and use Azure OpenAI for security.

Good question!

No, we actually don't even offer our own endpoints for inference yet, precisely because of the data concerns you site.

Right now it's self-host only, but it's very very easy to self-host, and we'll keep it that way for every feature touching user data.

Azure and security in one sentence.

Good one.

Yes but They sign BAA for HIPAA. OpenAI does not last we checked.

That's a business associate agreement, for people outside the HIPAA world.

https://www.hhs.gov/hipaa/for-professionals/privacy/guidance...

Nice project! I made Beak.js which has a similar way to connect a language model to the UI via React hooks: https://github.com/mme/beakjs

Great minds think alike!

Our `useMakeCopilotActionable` is similar to your `useBeakFunction` and `useMakeCopilotReadable` is analogous to your `useBeakInfo`.

I don't know how much farther you're planning to take things or if it's a demo/hobby project.

But real talk, if you're excited about this space and would like to explore joining our team, I'd love to chat and take it from there.

atai at copilotkit dot ai

I'm really liking the agnosticism with Beak.js. Would love to read a blog post about the power of that or examples of useBeakFunction use-cases

Relately: I've noticed that if I use ChatGPT in Microsoft Edge, I get (sometimes very) helpful co-pilot style completions in the input text area. Is that an Edge-exclusive feature? And how does CopilotKit compare?

I didn't know ChatGPT + Edge did that, pretty cool!

And yes, with CopilotKit you can implement the same to any text area in any app, that would work on any browser (+ other key features).

I can tell you that CopilotTextarea works very, very well if you give it the right context - but since I hadn't known about Edge's feature, I can't compare.

Even long term, I think this is functionality that will be better offered by the app itself rather than the browser - bc the app can control the relevant context beyond what's visible on the screen (e.g. RAG on users' backend state), customize the prompt, and potentially even design dedicated LLM chains (as needed).

So, wait, edge posts non-submitted content you've entered into textboxes to a cloud llm? or is this running in your browser?

I now think it's likely I misunderstood what the feature involves and that it is actually way more limited than I thought compared to CopilotKit (perhaps it's little different from Gmail's Smart Compose), but this is a quick-and-dirty screenshot:

https://imgur.com/a/FwZu6f7

This looks like a very rudimentary completion, almost Markov chain level. I don't think it requires sending user data to any remote servers, it could have easily been done on the client.

The question isn't how it could be done but how it is done.

Funny you find those useful, I disabled them fairly quickly.

That said, prior to disabling them I’d actually use it the opposite way: If I’m entering prose into a box and the AI has a very good understanding of what I’m about to say, that’s a good indication I’m not saying anything remotely interesting and I should either change direction or abandon my comment entirely.

Very interesting use-case!

With a creative project, if the AI can predict what you say next, then it's not creative enough!

So this has nothing to do with Copilot from GitHub?

Correct, Copilot here stands for AI "second brain" that "co-pilots" the app alongside the user.

(the abstract idea behind GitHub Copilot).

Well it’s kind of misleading, because GitHub has kind of a following for their product named Copilot.

I think renaming it would benefit everyone.

Microsoft also has an AI feature called Copilot. I think at this point the name has become a generic term for AI assistance.

> > Well it’s kind of misleading, because GitHub has kind of a following for their product named Copilot.

> Microsoft also has an AI feature called Copilot. I think at this point the name has become a generic term for AI assistance.

Github is Microsoft, and, yes, Microsoft has expanded their Copilot branding for end-user-facing AI assistance beyond just the Github coding tool, but that doesn't make it generic.

Grow your OpenAI bill exponentially with this one easy trick!

In all seriousness — looks nifty! Do you guys have any plans to bring suggestion latency down from what’s show in the demo?

Latency is actually very low as-is, with GPT-4-Turbo!

But other than that, yes, we do have plans - using real-time RAG. I carried out some research with incredibly promising results

https://ai88.substack.com/p/rag-vs-context-window-in-gpt4-ac...

It'd be nice if this was more clearly labeled as a React plugin. I saw "drop in replacement for <textarea>" and thought it was a web component and only realized it was a react plugin after reading further. Not a huge deal, but since it is a plugin and not a stand-alone thing it might make sense to make that clear up front.

Thanks for the feedback! I'll update the docs.

(also, I know that wasn't your core concern, but we are planning to build SDKs for other frontends in the not too distant future!).

Cool concept! Would this work with React Native, now or in the future?

We're starting with react web, planning to expand from there to other surfaces, including React Native.

I suspect the gap from the current version, to React Native support is rather small. It's open-source, so feel free to open an issue so that it can get upvoted/prioritized, (or even better, you can work on it too ;) )

Cool! Any plans for Svelte?

Super cool! Heads up: when clicking "website" next to "Join our Discord" on your repo, it looks like the link is broken

Thanks for the pointer, fixed!

This is great, if I understand correctly I just inject the hook - <CopilotPortal /> ?

Thanks!

And Pretty much! You have to wrap the app (or the part you want copilot in) with a <CopilotProvider /> - this is UI-agnostic, business logic only. Then there are a few customizable UI providers you can use, or you can just build your own if needed.

If you check it out do let me know what you think!

Amazing product! How does the connection to LLMs works? Bring your own LLM/API key?

Yes, exactly.

And thanks for the kind words!

Pretty cool tech!

Thank you!

Pretty cool idea

Thank you!

Cool idea!

Thanks!

Why use the Copilot name? It's confusing because you seem to have an agnostic product.

It's a fair point :)

AI Copilot started out synonymous with code completion, it still means that for many.

Our bet was this meaning would broaden with time.

Now that it’s firmly entrenched as Microsoft branding you may want to reconsider. Soon, people searching for “AI Copilot” aren’t going to be able to find you.

It's a fair point.

Remains to be seen if it'll become a common word, or Microsoft-associated.

Microsoft is using the Copilot term everywhere - in GitHub, Visual Studio, all the Office apps etc. They have rebranded the Power Platform Virtual Agents builder as "Copilot Studio", and chatbots in your projects are known as copilots.

I would be very surprised if you didn't receive legal challenge to your use of the name, if you grow in popularity.

If I see functionality like this on a company's website, I am going to find someone else to do business with about 80% of the time.

why? Doesn't this generally help you write faster?

Any chatbot/AI that i can't say "please refund this order due to product being damaged here is a pic" and it can't do a refund on its own is worthless. Not because i care about refunds, but if the bot can't do that, chances are it can't do anything real and its just trained on a bunch of docs and is worthless otherwise.

It's also 10x worse when the website has a green circle on it like teams/zoom showing it as "online" and makes you feel like you are actually talking to someone.

Obviously we'll let customers decide how to deploy this tech, but our vision is less about "chat with AI customer support", and more "tap into intelligence to help you achieve your task".

I personally fell in love with copilots when all my engineering tools started adding them (e.g. Cursor IDE, Warp terminal, etc.).

And really, I wanted to tap into artificial intelligence everywhere, from excel to planning a vacation in AirBnb. "is this a good place to stay with a young kid?"

Just in recent months, the market has validated this intuition, e.g. Microsoft Copilot products (think 2nd-brain-on-tap for e.g. PowerPoint, not support chat) have boosted productivity by 30%, with lots of other impressive metrics

---

If you do go back to the customer support use-case, I'd see the support agent's software using CopilotTextarea, to let them reply to 10x more messages, but where they still drive the conversation.

Though I could see it coupled with AI chatbot on the consumer side, where it makes sense

It should not be cumbersome to enter your query and be suggested a solution before being connected to a human. In some cases the suggestion will resolve the problem, relieving the load on responders.

as a consumer i totally agree

but having been on the other side, requests like yours are the exception. support chatbots relieve a huge load, allowing the real people to work on real requests like yours rather than the numerous junk ones.

It's a waste of time.

Chat interface is used to talk to people. If you talk to a machine, there is no point to mimic human behaviour, especially when current iterations of AI can't reason. There should be a better way to do this.

Like 90% of the time I want to chat to customer rep, because I want to let them now there were missing items in the delivery and I want re-delivery or refund. That could be a simple expert system with a few buttons. But businesses want to discourage people from returning anything, so they are building those time wasting widgets hoping a % of people give up.

Like if my time costs £200 per hour and the wrong delivery was worth £50, I am not going to spend more than 15 minutes on this, unless it is critical to the business.

This has nothing to do with helping customers, but to save money.

I responded to a sibling comment of yours about the chat aspect (TL;DR: our key objective is very far from human customer support - more similar to "chat with your codebase" if you use e.g. Cursor IDE).

But I think you do bring an interesting point, that chat interface is just one type of way to talk to computers, and in some sense it's the "laziest" way, since it's what we use to talk to people now.

The Textarea is actually an example of another non-chat way to "talk" to the AI intelligence. I do think we'll see a lot more being discovered by the community over the next 2 years (and we'll build those into CopilotKit, of course)

Feels like Microsoft might send you a letter soon about the nane...

How about FirstOfficerKit, but that seems very far away from AI if one doesn't get that it's a synonym for copilot

It feels like MS is already treating it like a generic term. I think I saw a Kevin Scott article recently where he kept referring to "copilots" as well. The term is being so widely used right now that they'd need to defend it quickly if they even have a trademark. (Looking at the USPTO database, MS applied for a trademark for "Microsoft Copilot" in September then filed an express abandonment a few days later.)

Yeah, I hope that doesn't happen.

That said the concept of a copilot is pretty generic, so hopefully we can peacefully coexist.

They've doubled down on the Copilot naming after the project was well underway, and I don't really want to rename it in anticipation of trouble that may never materialize

The term is just blatantly lifted from an entire other industry, can you really trademark that?

It's an airline term, I am not a lawyer at all but I feel like there's an argument somewhere that using it as a brand for an intelligent/computer assistant, or, fine if you want to be buzzwordy, AI assistant on a computer is a novel use.

A bit like "Apple". If you open a computer store named Apple, I think you can't get away with the argument that "It's a term from the agriculture industry!".

Open-Source (MIT).

If you'd like to contribute to the repo, or to the open copilot protocol, please join the GitHub/Discord!

Here's an open source alternative with context retrieval and message storage: https://github.com/get-convex/convex-ai-chat

(there are also versions using Assistants API and LangChain, linked from here: https://stack.convex.dev/ai-chat-using-openai-assistants-api)

CopilotKit itself is open-source, and supports assistants API + langchain :) https://github.com/CopilotKit/CopilotKit

Microsoft recently rebranded their chatbot offering to Copilot Studio. Copilot is also the name of Microsoft's AI products.

It's hard enough keeping all of Microsoft's products with similar names straight. I don't want to confuse other products with them, too, unless, maybe, this is the point :)

Awesome project. I was looking through the Github and if I understand correctly, then it's plugged in to the OpenAI API by default?

Is there an option for using other LLMs with it?

Being able to plug this into Google Docs using your own local LLM would be very interesting...

Correct.

The default backend implementation uses OpenAI, but you can use any LLM pretty easily.

Awesome, thanks!

[dead]