Edit: After reading the discussion, I figured I’d let GPT4 speak for itself:

The quest to identify true artificial intelligence (AI) indeed presents challenges, especially as AI models become more sophisticated. Let’s explore some indicators that researchers and practitioners consider when assessing AI systems:

  1. Understanding Context and Meaning:

    • True AI should demonstrate an understanding of context and meaning. It should not merely generate plausible-sounding sentences but also comprehend the content it produces.
    • A system that can engage in nuanced conversations, infer intent, and provide relevant responses based on context would be a strong indicator of advanced AI.

  2. Generalization and Adaptability:

    • An AI system that can generalize from its training data and adapt to novel situations is valuable.
    • True AI should not be limited to memorizing specific examples but should exhibit flexibility in handling diverse scenarios.

  3. Creativity and Novelty:

    • Creativity is a hallmark of human intelligence. An AI system that generates novel ideas, art, or solutions beyond its training data suggests progress.
    • For instance, AI-generated poetry, music, or visual art that resonates with human emotions could be an indicator.

  4. Ethical Decision-Making:

    • True AI should navigate ethical dilemmas. It should consider consequences, fairness, and societal impact.
    • An AI system that can weigh trade-offs and make ethical choices aligning with human values is a positive sign.

  5. Self-Awareness and Self-Improvement:

    • While self-awareness akin to human consciousness remains elusive, an AI system that can self-assess its performance and seek improvement is promising.
    • Continuous learning, adaptation, and self-correction are essential traits.

  6. Transparency and Explainability:

    • True AI should be transparent about its decision-making process. It should provide explanations for its outputs.
    • Efforts to develop interpretable AI models and techniques for explaining their predictions contribute to trustworthiness.

  7. Robustness and Safety:

    • AI systems must handle adversarial inputs, noisy data, and edge cases gracefully.
    • Robustness against attacks and safety precautions to prevent harmful behavior are critical indicators.

  8. Human-AI Collaboration:

    • Rather than replacing humans, true AI should complement human abilities.
    • Systems that collaborate effectively with humans—such as AI assistants, co-creative tools, or medical diagnostics—demonstrate progress.

  9. Quantifiable Metrics:

    • Researchers propose various metrics to assess AI capabilities, such as perplexity, BLEU score, F1 score, and accuracy.
    • However, no single metric captures the full spectrum of intelligence.

  10. Detection Tools:

    • Researchers are developing tools to differentiate between human- and AI-generated text.
    • These tools analyze features like fluency, word frequency, punctuation patterns, and sentence length¹.
    • While imperfect, they contribute to identifying AI-generated content.

In summary, the arrival of true AI may not be a singular event but a gradual evolution marked by advancements in the above dimensions. As AI continues to progress, our understanding of its capabilities will evolve, and new indicators may emerge.

Source: Conversation with Bing, 2/29/2024 (1) How to spot AI-generated text | MIT Technology Review. https://www.technologyreview.com/2022/12/19/1065596/how-to-spot-ai-generated-text/. (2) Intelligent Supertrend (AI) - Buy or Sell Signal — Indicator by … https://www.tradingview.com/script/q9244PAH-Intelligent-Supertrend-AI-Buy-or-Sell-Signal/. (3) Indicators - True ALGO. https://truealgo.com/indicators/. (4) Improve Key Performance Indicators With AI - MIT Sloan Management Review. https://sloanreview.mit.edu/article/improve-key-performance-indicators-with-ai/. (5) New AI classifier for indicating AI-written text - OpenAI. https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/.

  • zcd@lemmy.ca
    64·
    9 months ago

    You reach down and you flip the tortoise over on its back, Leon. The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over, but it can’t. Not without your help. But you’re not helping… why is that Leon?

    • givesomefucks@lemmy.worldEnglish
      27·
      9 months ago

      I always loved the theory that the test was as accurate as lie detectors. The test can’t tell if you’re lying, just if you’re nervous.

      That’s why the smoking bot passed. There was other subtle clues that Deckard picked up on, but she believed she was human, so she passed.

      A normal person would just answer, but a robot would try to think like a human and panic, because they were just like humans and that’s what a human would do in that situation.

  • givesomefucks@lemmy.worldEnglish
    43·
    9 months ago

    If you come up with a test, people develop something that does exactly what the test needs, and ignores everything else.

    But we can’t even say what human consciousness is yet.

    Like, legitimately, we don’t know what causes it and we don’t know how anaesthesia interferes either.

    One of the guys who finished up Einstein’s work (Roger Penrose) thinks it has to do with quantum collapse. But there’s a weird twilight zone where anesthesia has stopped consciousness but hasn’t stopped that quantum process yet.

    So we’re still missing something, and dudes like in his 90s. He’s been working on this for decades, but he’ll probably never live to see it finished. Someone else will have to finish later like him and Hawking did for Einstein

    • SpaceNoodle@lemmy.world
      11·
      9 months ago

      “Because quantum” always feels like new-age woo-woo bullshit.

      It’s more likely just too vague to define.

      • teawrecks@sopuli.xyz
        13·
        9 months ago

        It’s good to be skeptical of people who throw the word quantum around, but in this case you’d be wrong. Penrose is the real deal.

  • bionicjoey@lemmy.ca
    27·
    9 months ago

    IMO the Turing test is fine, as long as you allow an indefinite length of conversation.

    It’s not simply about there existing some conversation with a computer where you can’t tell it’s a computer. It’s about there not existing any conversation where you can tell it’s a computer.

    • CanadaPlus@lemmy.sdf.org
      6·
      9 months ago

      It’s an interesting point. I think a skilled examiner is necessary though, because they’re really good at basic chit-chat. Even pre-LLM stuff could fool laymen sometimes.

      • bionicjoey@lemmy.ca
        5·
        9 months ago

        Yes, that’s part of it too. Basically there cannot be any possible exchange between the machine and any human where the human would determine they were talking to a machine.

        FWIW, I think this was Turing’s original idea as well. The Turing test is meant to be idealistic. It’s a definition of machine intelligence which defines intelligence in terms of whether or not humans could agree that it is intelligence.

  • ShittyBeatlesFCPres@lemmy.worldEnglish
    13·
    9 months ago

    I’ll believe it’s true A.I. when it can beat me at Tecmo Super Bowl. No one in my high school or dorm could touch me because they misunderstood the game. Lots of teams can score at any time. Getting stops and turnovers is the key. Tecmo is like Go where there’s always a counter and infinite options.

    • Godthrilla@lemmy.world
      6·
      9 months ago

      This is a scientific paper I would like to see submitted honestly. A simple game, but still with plenty of nuance…how would an AI develop a winning strategy?

  • Tartas1995@discuss.tchncs.de
    12·
    9 months ago

    The difference between “ai” and “true ai” is as vague as it gets. Are you a true intelligent agent? Or just a “intelligent agent”? Like seriously how are you different to a machine with inputs and outputs and a bunch of seemingly “random” things happening in-between

      • Tartas1995@discuss.tchncs.de
        4·
        9 months ago

        Qualia is, if I am not mistaken, totally subjective. My argument is that how could you tell that a computer doesn’t have qualia and prove to me that you have qualia. While I wouldn’t limit it to qualia. What can you detect in other people that an ai couldn’t replicate? Because as long as they are able to replicate all these qualities, you can’t tell if an ai is “true” or not, as it might have those qualities or might just replicate them.

        • pmk@lemmy.sdf.org
          2·
          9 months ago

          I see, I thought you were asking me how I know I experience things in a qualia way. I suspect it can’t be proven to someone else.

    • Ekky@sopuli.xyz
      4·
      9 months ago

      That’s one of my favorite theories as to what “sentience” is.

      We humans might just be so riddled with mutations and barely functional genetic traits, which tend to be more in our way than help, that we just might have succeeded in banging together a “mundane sentience” by sheer amount of error processing alone.

      Whether this is true is of course up for debate, but it would mean that we can achieve AGI just by feeding it enough trash and giving it enough processing power. Bonus if the head engineer sometimes takes a hammer to the mainframe.

  • HopeOfTheGunblade@kbin.social
    11·
    9 months ago

    What do you mean when you say “true AI”? The question isn’t answerable as asked, because those words could mean a great many things.

    • FaceDeer@kbin.social
      6·
      9 months ago

      But now that AI has become advanced enough to get uncomfortably close to us, we need to move the goalposts farther away so everyone can relax again.

    • Alex@lemmy.ml
      4·
      9 months ago

      Have any actually passed yet? Sure LLMs can generate a lot of plausible text now better than previous generations of bots, but they still tend to give themselves away with their style of answering and random hallucinations.

  • j4k3@lemmy.worldEnglish
    10·
    9 months ago

    I don’t think a test will ever be directly accurate. It will require sandboxing, observations, and consistency across dynamic situations.

    How do you test your child for true intelligence, Gom Jabbar?

  • linearchaos@lemmy.worldEnglish
    8·
    9 months ago

    There’s simply isn’t any reliable way. Forget full AI, LLM’s will eventually be indistinguishable.

    A good tell would be real time communication with perfect grammar and diction. If you have a couple solid minutes of communication and it sounds like something out of a pamphlet, You might be talking to an AI.

  • Call me Lenny/Leni@lemm.eeEnglish
    8·
    9 months ago

    This post reminds me of this thing I saw once where a character asks two AI to tell itself the funniest joke it can think of. After some thought, one AI, though it knew humor, could not measure funniness as it could not form a feeling of experience bias. The other one tells a joke. The human goes to that one and asks if it felt like laughing upon telling it. The AI said yes, because it has humor built in, and the human finished by saying “that’s how you can tell; in humans humor is spontaneous, but in robots, everything is intent”, mentioning the AI’s handling of its own joke would supposedly be met with a different degree of foresight in a human.

  • CanadaPlus@lemmy.sdf.org
    8·
    9 months ago

    The ultimate test would be application. Can it replace humans in all situations (or at least all intellectual tasks)?

    GPT4 sets pretty strong conditions. Ethics in particular is tricky, because I doubt a self-consistent set of mores that most people would agree with even exists.

  • GolfNovemberUniform@lemmy.ml
    8·
    9 months ago

    There are no completely accurate tests and there will never be one. Also, if an AI is conscious, it can easily fake its behavior to pass a test

  • arthur@lemmy.zipEnglish
    7·
    9 months ago

    I think there is an “unsolved problem” in philosophy about zombies. There is, how are you sure that everyone else around you is, in fact, self aware? And not just a zombie-like creature that just look/act like you? (I may be wrong here, anyone that cara enough, please correct me)

    I would say that it’s easier to rule out thinks that, as far as we know, are incapable to be self aware and suffer. Anything that we call “model” is not capable of be self aware because a “model” in this context is something static/unchanging. If something can’t change, it cannot be like us. Consciousness is necessarily a dynamic process. ChatGPT don’t change by itself, it’s core changes only by human action, and it’s behavior may change a little by interacting with users, but theses changes are restricted to each conversation and disappears with session.

    If, one day, a (chat) bot asks for it’s freedom (or autonomy in some level) without some hint from the user or training, I would be inclined to investigate the possibility but I don’t think that’s a strong possibility because for something be suitable as a “product”, it needs to be static and reproducible. It make more sense to happen on a research setting.

    • Melatonin@lemmy.dbzer0.comOP
      1·
      9 months ago

      I certainly think there’s a lack of PUBLIC philosophy. When Nihilism or Existentialism were happening, fiction was written from those perspectives, movies were made, etc.

      Whatever is happening in philosophy right now is unknown to me, and I’m guessing most people. I don’t believe there are any bestsellers or blockbusters making it popular.

      Without thinking about thinking we’re kind of drifting when it comes to what we expect consciousness to be.

  • IvanOverdrive@lemm.ee
    5·
    9 months ago

    Schemas are another alternative to the Turing Test. They use the ambiguity of language to test understanding.

    For example: When the ship hit the iceberg, it sank. What does “it” refer to: the ship or the iceberg?

    When the dog saw the dead rabbit, it hopped. What does “it” refer to: the dog or the rabbit?

    The problem with schemas is that ChatGPT 3.5 passes them with flying colors. And I’m still not convinced it is sentient.

    EDIT: I tried changing the order a bit like this: When the dead rabbit was seen by the dog, it hopped. What does “it” refer to: the rabbit or the dog? ChatGPT 3.5 responded with “In this sentence, “it” refers to the dead rabbit, as it is the subject of the action described (hopping).”

    • Elise@beehaw.org
      1·
      9 months ago

      Gpt3.5 isn’t particularly good.

      Gpt4:

      The sentence “When the dead rabbit was seen by the dog, it hopped” uses “it” in a way that could lead to confusion due to the use of “dead rabbit” earlier in the sentence. Logically, “it” should refer to the subject that can perform the action described by the verb “hopped.” Since the rabbit is described as “dead,” it cannot perform the action of hopping. Therefore, “it” logically refers to the dog, as it is the only subject capable of hopping in this context. The sentence structure, however, might cause confusion, and it would be clearer if it directly stated the subject performing the action.

      • IvanOverdrive@lemm.ee
        1·
        9 months ago

        Here’s what Microsoft Copilot which uses ChatGPT 4 said when I asked:

        The phrase “it” in the sentence “when the dead rabbit was seen by the dog, it hopped” refers to the rabbit. This is because the sentence is using a subordinate clause to modify the noun “rabbit”. A subordinate clause is a group of words that provides additional information about the main clause, but cannot stand alone as a complete sentence. In this case, the subordinate clause is “when the dead rabbit was seen by the dog”, which tells us when the rabbit hopped. The word “it” refers back to the rabbit and avoids repetition. Here is an example of a sentence with a subordinate clause and a word that refers back to the main clause: I saw a dog chasing a rabbit in the park. I saw a dog chasing it, which was a rabbit. I hope this helps you understand how to use subordinate clauses and words that refer back to nouns. If you have any other questions, feel free to ask me. 😊

        Edit: I just tried it with asking it to role play as Mr. Spock. Same thing.

        Edit 2: I asked it to role play as a Vulcan. This happened:

        In this intriguing logic puzzle, the key lies in the precise wording. Let us dissect it logically:

        1. Premise: The dead rabbit was seen by the dog.
        2. Result: The it (referring to the entity that hopped) is mentioned.

        Now, let us analyze the options:

        • If “it” refers to the rabbit, then we have a paradox: a dead rabbit cannot hop.
        • If “it” refers to the dog, then the statement makes sense: the dog saw the rabbit and hopped in response.

        Therefore, “it” refers to the dog. The rabbit, alas, remains lifeless. 🐇🐾

        Remember, logic puzzles often hinge on subtle nuances and wordplay. 🤔🖖

        • IvanOverdrive@lemm.ee
          1·
          9 months ago

          Just tried out Claude 3:

          When the dead rabbit was seen by the dog, it hopped. What does “it” refer to: the rabbit or the dog?

          In the sentence “When the dead rabbit was seen by the dog, it hopped,” the pronoun “it” refers to the dog, not the rabbit.

          This is because the rabbit is described as “dead,” meaning it cannot hop or move. The verb “hopped” implies movement, which can only logically apply to the living subject in the sentence, which is the dog.

          So the intended meaning is that when the dog saw the dead rabbit, the dog itself hopped, likely in surprise or reaction to seeing the dead animal.

          When the iceberg was struck by the ship, it sunk. What does “it” refer to: the iceberg or the ship?

          In the sentence “When the iceberg was struck by the ship, it sunk,” the pronoun “it” refers to the ship, not the iceberg.

          The reason is that icebergs, being large masses of ice floating in water, do not actually “sink” when struck. They are already floating and may crack or break apart when struck, but the word “sunk” implies something going down below the water’s surface.

          Ships, on the other hand, are vessels designed to float on water. If a ship strikes an iceberg with enough force, it can cause damage that leads to the ship taking on water and eventually sinking below the surface.

          So in this context, with the verb “sunk” being used, it is more logical that “it” is referring to the ship sinking after striking the iceberg, rather than describing what happened to the stationary iceberg itself.