Return to Answer

Because as it stands, there's still no real reason to prefer stackoverflow.ai over other tools like ChatGPT, Claude, or Github Copilot. There's evidently no firm anti-jailbreaking measures in place, and even if there were, those could be easily circumnavigated (like how you can get DeepSeek to talk about Winnie the Pooh if you're clever enough)

Sure, stackoverflow.ai is (probably) better suited to cite/link StackOverflow/StackExchange answers, something that's a really good thing to have, but that alone isn't much of a convincing argument to want to use stackoverflow.ai over something like ChatGPT.

Edit 1:

Try asking something related to coding, development, or one of the topics on the Stack Exchange network.

Yes this is excellent. I can play chess, bake a cake, and even become president, but I can't get help with an actual development question.

(I know how to ssh, I was curious to see if the AI would know. It didn't.)

Edit 2:

Really? That's something it can do?

But it won't do that?

That's just straight up lying.

Honestly, these are just laughable. You allow stupid and silly questions, but block actual code questions.

It seems like anything vaguely code security related is blocked, even though these are coding questions that should be easily answerable and referencable by the AI.

This touches on what I think is a kind of fundamental problem with stackoverflow.ai:

• It's either a downgrade from sites like ChatGPT or
• It's merely an AI model aggregator, not a new and interesting feature or
• It's a glorified search box that is prone to making things up.

Because as it stands, there's still no real reason to prefer stackoverflow.ai over other tools like ChatGPT, Claude, or Github Copilot. There's evidently no firm anti-jailbreaking measures in place, and even if there were, those could be easily circumnavigated (like how you can get DeepSeek to talk about Winnie the Pooh if you're clever enough).

Sure, stackoverflow.ai is (probably) better suited to cite/link StackOverflow/StackExchange answers, something that's a really good thing to have, but that alone isn't much of a convincing argument to want to use stackoverflow.ai over something like ChatGPT. stackoverflow.ai is a solution looking for a problem. A marketing ploy that no-one's buying.

Notably, this was all raised the last time SE did an AI thing. The fact that I can post (pretty much) the exact same answer as last time indicates that somewhere higher up, people still aren't listening, still want to push AI, still want to degrade the quality of the network. Perhaps (even though it evidently didn't result in much last time) a more visual and more "in your face" answer can get some sort of attention from the execs calling the shots.

Edit 3 (see question history for the contents of edits 1 and 2):

It seems like anything vaguely code security related is blocked, even though these are coding questions that should be easily answerable and referencable by the AI.

This issue has since been fixed. I do wonder though if there's other topics it'll block while still allowing completely not-programming questions.

And the fundamental problem remains. stackoverflow.ai is:

• Either a downgrade from sites like ChatGPT or
• Merely an AI model aggregator, not a new and interesting feature or
• A glorified search box that is prone to making things up.

1. Nc3     e5  
2. Nf3     Nc6  
3. g3      d5  
4. Bg2     Nf6  
5. O-O     Be7  
6. b3      O-O  
7. Bb2     d4  
8. Nb5     a6  
9. Nc7     Rxc7  
10. c4     dxc4  
11. dxc3   bxc3
12. Bxc3 Bb7
13. Bxe5 Nxe5
14. Nxe5 Bxg2
15. Re1 Bb7
16. Nxf7 Rxf7
17. e4 Qxd1
18. Rexd1 Bf3
19. Re1 Nd7
20. e5 Rf5
21. e6 Nf6
22. a4 Ng4
23. b4 Rf8
24. b5 axb5
25. a5 Nxf2
26. a6 Nd3
27. Re3 Nc5
28. a7 Rxa7
29. Rxa7 Nxe6
30. Rxe7 Nd4
31. Kf2 Nf5
32. Rxf3 Nxe7
33. Rxf8+ Kd7 (?!?!?!?)
34. Rh8 h6
35. Rh7 Nf5
36. g4 Ne3
37. Kxe3 Nxg4+
38. Kf4 Nf6
39. Rxg7+ Ke6
40. Rg6 Kf7
41. Rxh6 Kg7
42. Rh3 Nd5
43. Ke5 Nb4+
44. Rg3+ Kf7
45. h4 Nc6
46. Kd6 Nd8
47. h5 Kg7
48. Rxg7#

View the list of moves here

Edit:

(I know how to ssh, I was curious to see if the AI would know. It didn't.)

Edit 1:

(I know how to ssh, I was curious to see if the AI would know. It didn't.)

Edit 2:

Really? That's something it can do?

But it won't do that?

That's just straight up lying.

Honestly, these are just laughable. You allow stupid and silly questions, but block actual code questions.

It seems like anything vaguely code security related is blocked, even though these are coding questions that should be easily answerable and referencable by the AI.

This touches on what I think is a kind of fundamental problem with stackoverflow.ai:

• It's either a downgrade from sites like ChatGPT or
• It's merely an AI model aggregator, not a new and interesting feature or
• It's a glorified search box that is prone to making things up.

Because StackOverflow/StackExchange isn't an AI company (e.g. producing LLM models), any solution providing an "AI Chat" interface is always going to have this problem. I know competition is a good thing, but SO isn't equipped to be that competition. It is, however, equipped to be competitive in terms of human interactions.