Edit - Meta Stack Exchange

1. Home
2. Questions
3. Unanswered
4. Tags
6. Chat
7. Users
Stack Internal

Stack Overflow for Teams is now called Stack Internal. Bring the best of human thought and AI automation together at your work.
Try for free Learn more
Stack Internal
Bring the best of human thought and AI automation together at your work. Learn more

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

Rev

Required fields*

Labs experiment launch: stackoverflow.ai

UPDATE - July 9, 2025

This week we’ve updated stackoverflow.ai with the following new functionality and fixes:

Related content suggestions are more relevant and dynamically re-ranked during conversational chat
Related content suggestions can now come from the Stack Exchange network, in addition to Stack Overflow
UX updates, improvements and fixes
All users now see the link on the left navigation menu, which is now “AI Assist”

Still pending release later this week:

Import chat history option allows for content from other LLM conversations to be imported via summary
Improved mobile accessibility for the chat experience
Built-in feedback mechanism

Many thanks to those who’ve provided feedback and shared thoughts on this post. Since the experiment now has a built-in feedback mechanism, if you are engaging with the experiment you can use that to provide feedback about your experience. General feedback remains welcome here.

Some of you asked very valid questions about who this experiment is targeting and whether this concept can be a new on-ramp into the community. The reason for this experiment is to explore the potential audience and entry path.

This experiment exists in the broader evolving landscape around LLM attribution. We acknowledge the concerns around attribution of the chat responses, and we are working to address this in the best way for the community and users.

Continuing experimentation around themes of reaching and supporting technologists and smarter discovery, today (June 25, 2025), we're announcing a limited experiment on stackoverflow.ai, a new AI-powered search and discovery tool.

What is stackoverflow.ai?

We’ve experimented with AI-powered search and discovery before, so what’s different this time? Past concepts were RAG-based (Retrieval-Augmented Generation) and simply surfaced answers from Stack Overflow. The stackoverflow.ai experiment offers a model-agnostic generative AI tool, trained on knowledge from the broader web (including the Stack Exchange Network). As the user interacts with the tool, related content from Stack Overflow is displayed in the sidebar. This human-authored content from Stack Overflow is available as an entry point into the community and can help the user validate output from the genAI conversation.

The goal is to provide users with:

A new way to get started on Stack Overflow. The tool can help developers get unblocked instantly with answers to their technical problems, while helping them learn along the way and providing a path into the community.
A familiar, natural language experience that anyone who has interacted with genAI chatbots would expect, but further enriched with clear connections to trusted and verified Stack Overflow knowledge.
A user-friendly interface with conversational search and discovery.
A path, when the genAI tool isn’t providing the solution they need, to bring their question to the Stack Overflow community via the latest question asking experience, including Staging Ground.

This limited release is a first iteration to understand infrastructure viability, identify and fix bugs, assess core functionality, and gather initial feedback before considering opening it up to more testing and adding more functionality.

Additionally, this limited release will help us ensure our tools effectively detect and manage unrelated, inappropriate, or harmful content. It is important that we get this right, so if you get responses from this feature that are incorrect, harmful, or otherwise, inappropriate, please contact our team by using the "Contact" link here, selecting Trust and Safety under “What can we help you with?”, and selecting I have a concern with StackOverflow.ai.

What comes next?

Over the next few weeks, we will assess core functionality and gather initial feedback from the community and the randomly selected users and visitors using the feature.

Provided this early testing phase goes well, in July, we expect to add the following features and capabilities:

Import chat history - Developers can pick up right where they left off in another AI tool to get unstuck on stackoverflow.ai.
Related content suggestions from the Stack Exchange network, as well as Stack Overflow.
Dynamic re-ranking of the related content based on the ongoing genAI conversation.
The path to post a question directly to the relevant Stack Exchange site.
Additional ways to provide feedback and flag content on the genAI response.

This post is for bug reports and suggestions from users who have tried out the new interface, as well as for general feedback from the Meta community — how might this evolve to do more for developers, or for you?

Answer*

The new experiment is an off-the-shelf LLM chatbot without attribution, with StackOverflow advertisements on the side and a link to the Ask a question page.

Your goal seems to be to make it easier to establish StackOverflow as an alternative for all the questions that cannot be sufficiently answered by an LLM or related questions on this platform. You also ignore your own commitment to attribution.

In this regard the future of the exchanges would be to answer those questions that cannot be answered by chatbots or existing content. You seem to acknowledge that people use chatbots nowadays relatively abundantly, and so you hope that if you can't beat them, you can at least sneak into the process a bit and in the best case add some value or maybe just divert some traffic.

I'm not sure I will use it a lot. Nowadays I have the choice to use LLM chatbots directly (often a bit more convenient), use search engines (offer links from a larger universe not just SO) or use this platform directly, additionally to this new service which mingles everything a bit. I don't see a clear advantage of the experiment.

And that is before taking into account the abysmal quality of the SO advertisement links on the right side. They are not fitting at all.

One example:

> I tried an ontopic question "How to create a multipage app with dash?" I expected the LLM answer to be good (it was) and the right side links to be relevant (they weren't). One was about jquery, one about a debugging problem with dash, one about tumblr and javascript and that's while at least two relevant questions exist. Googling for the question delivers more relevant links outside of SO. The combination of using an LLM chatbot directly + googling directly seems like the better approach currently.

This feature could maybe have some value if the quality of the target links on the right side is increased tremendously. I would make that highest priority. For example you could add some feedback buttons to the proposed SO Q&As like this link is/is not helpful and is-not would remove it and maybe load something else, then learn from the feedback.

----

P.S.:

> This experiment exists in the broader evolving landscape around LLM attribution. We acknowledge the concerns around attribution of the chat responses, and we are working to address this in the best way for the community and users.

So you keep saying but so far nothing has materialized yet, while you offer the service already. That's not only a breach of your commitment, it might also be dishonest, promising to work on it again and again but having nothing to show for it. You say it yourself, the used chatbot is trained on our data, but we are not given any attribution, not even here. You said it's not negotiable but that must have been not true.

Edit Summary*

Cancel

3

Thank you for the feedback and we're thinking the same thing: the value is dependent on the related content (and actions to come) that can supplement the LLM experience. The alpha is not there and this was known to us, but your post and several others are helping us test the concept. I'm looking forward to the next few weeks as we improve the related content relevance.

Ash Zade
– Ash Zade Staff

2025-06-27 11:57:57 +00:00
Commented Jun 27 at 11:57
2

@AshZade nevermind the (nonexistent) value, the contract SE has with the community is being broken by not providing proper attribution. What is your plan to rectify that ongoing breach of trust which goes against direct assurances given by the company?

l4mpi
– l4mpi

2025-06-27 12:35:47 +00:00
Commented Jun 27 at 12:35
2

My colleague posted an answer re: attribution here meta.stackexchange.com/a/411009/1258352

Ash Zade
– Ash Zade Staff

2025-06-27 12:41:06 +00:00
Commented Jun 27 at 12:41
16

@AshZade that answer calls attribution a "key value" that is required for "ethical AI use". And then goes on to say oh well seems like the LLMs are not there yet. Do you see the issue here? I am not asking for a BS PR statement that states how "key" this is for SE and gives some vague assurance of future improvements. I am asking why you are currently running a service which, according to the answer you linked, is unethical.

l4mpi
– l4mpi

2025-06-27 12:55:28 +00:00
Commented Jun 27 at 12:55

Add a comment |

Correct minor typos or mistakes
Clarify meaning without changing it
Add related resources or links
Always respect the author’s intent
Don’t use edits to reply to the author

create code fences with backticks ` or tildes ~
```
like so
```
add language identifier to highlight code
```python
def function(foo):
print(foo)
```
put returns between paragraphs
for linebreak add 2 spaces at end
_italic_ or **bold**
indent code by 4 spaces
backtick escapes `like _so_`
quote by placing > at start of line
to make links (use https whenever possible)

<https://example.com>

[example](https://example.com)

<a href="/api/flow.js?q=https%3A%2F%2Fexample.com">example</a>

formatting help »
answering help »