Chris Heuer (00:00) So my reliance on AI being right and trusting it led to a cascade of effects that we later had to unravel. And I basically reproduced all of the content around the episode, all of the clips, and spent a lot of time fixing that.
And so what arose was our desire to express it, to talk about it, and to use it as a teaching example,
Marius (00:18) So
Chris Heuer (00:19) as opposed to just ignoring it and pretending it didn’t happen.
Marius (00:21) sometimes the humans are responsible to telling the AI to create the slop.
Chris Heuer (00:25) but that brings us to a more important question around here that gets us into the show for the day, which is what is AI slop?
Good morning, good afternoon, good evening, wherever in the world you might be. Welcome to Community Assistant.ai, the podcast that’s helping community managers figuring out how to use AI as your assistant to get more done with less and help you do the work that you uniquely can do as a human. Building relationships, building trust within the community, and essentially making judgment about what AI slop is and isn’t. Anyways, we’ll get to that more
Marius (01:05) So.
Chris Heuer (01:27) in a minute. I’m joined today by my co-host, Marius Ciortea Marius, so very good to see you. How are you doing today?
Marius (01:34) Good, how are you Chris? Good, glad to be here again.
Chris Heuer (01:37) I am too. it’s been a a heck of a couple of weeks since we actually got that first pilot out. we’ll talk about that here in a more in a minute.
Well what one of the things that I found was that as we were doing the production of our pilot episode, which was really a trial run to make sure we were, you know, getting everything right and dialed in and made a bunch of changes since that, was that we were relying on a slightly mis represented quote that Marius
Marius (02:03) yeah.
Chris Heuer (02:04) had found. And as a result of that, we made the emphasis of the entire show and episode in the blog post on this idea that humans trust AI and AI trusts humans.
And it wasn’t exactly right. Marius could you help with the clarification on what we were actually getting at?
Marius (02:18) Well, you know,
like we said, like I was not exactly right, but it was not exactly wrong. think the study was quoting that AI or humans are now preferring in that word implied to the trusting AI Google results over regular search results.
Chris Heuer (02:35) Well, and and at the same time, we’re getting at like the core of this issue is that it’s it is while some people will trust it more because their experience is that they’re getting accurate answers, some people trust it less. And so what’s at stake here when we talk about AI slop and the use of AI in communities broadly is trust. And it’s one of these things, of course, we’ve talked about a lot that takes a long time to earn and can be lost in an instant.
So this is why AI Slop is the focus of this next episode and what we’re going to be talking about today.
which is what is AI slop?
Marius, you want to start with a run at that? I’ve got some things to add, of course, but I’d love to hear what your thoughts are and just briefly explaining what AI Slop is.
So that we can start with a common definition.
Marius (03:19) To me, it’s like when there is a simple answer, but instead of getting a simple answer, I get something extremely long and detailed. And there are some points that keep repeating themselves. And there are some some words in there that seem to be
used in other articles over and over again. so AI Sloppy is, is the difference to me is like from getting an answer that is actually useful to an answer that there’s something useful in there, but it will take me too long to figure out what exactly that answer is. So it’s become sort of like sloppy and,
sometimes the humans are responsible to telling the AI to create the slop. Maybe they need to refine their prompts to get us a little bit faster to the point. That’s a long-winded answer to a simple question. It’s sort of sloppy and it’s AI slop.
Chris Heuer (04:07) Well, it’s not that s it’s not that simple.
It’s not that simple. And But the way that I’ve actually rolled it down is about whether or not we bow down to slop or not.
And and bow in this case being is it bloated? again, you get that 8,000 words when you’re asking somebody for an answer to something and they send you like a full claude-generated AI response. the second element is is it off? does it read wrong? Is it using those common AI phrases, it’s not this, but it’s that, and things that recognize itself as AI writing, where you can say, that looks like Claude, that sounds like Claude. And the other one, which you were really hitting at, and I think is the most important one, is is it wrong?
And of course, the problem is that it presents itself in such a way that it sounds right, even when it’s presenting wrong information. So people without experience
Marius (04:55) you
Chris Heuer (04:56) in a given topic will believe it’s true and then continue to represent it as their own, which gets me into the final one, which is the intention. And the intention component
Marius (05:03) Okay.
Chris Heuer (05:05) of slop is that many people are just using AI as an easy button.
Marius (05:08) So if I’m understanding you right, you’re saying that can be AI writing that’s not slop.
Chris Heuer (05:14) Absolutely. Well, and you
Marius (05:15) Okay.
Chris Heuer (05:16) know, I’m a big fan of of Matt Mullenweg and WordPress and what they done. And you know, when you’re looking at code, his perspective is code is poetry. So for him, the process of writing code is craft, is art in and of itself. And so some people are still process oriented instead of outcome oriented, right? And I you know, and that’s actually the thing what’s leading into this article this week, the idea that slop is in the eye of the beholder.
You know, we all have slightly subjective opinions on what is slop, which is why we need to really, especially as community managers, define what slot is in our context, in our community, in our own personal use of it, and make that stand. And we’re not all going to agree. And and that’s generally fine. but it’s really helpful if we actually
Marius (06:01) Yeah.
Chris Heuer (06:01) do have a common definition of what it is. And like you, I don’t believe it’s the process of using AI.
and in fact, I I think I want to go to this op-ed piece that you were referring about, if you if we could talk a little bit more about that.
And the desire now of some people to just go to AI, write with all their writing, and just put it out there as is without even editing it. And I think that that is where you put yourself most in danger. If you don’t apply
Marius (06:33) you
Chris Heuer (06:26) human judgment to the things that you’re producing and sharing and writing and publishing with others, that’s where you increase the chance of slop
and where you increase the chance of losing trust. So what what what happened with this op-ed piece and the hedge fund management that you were talking about?
Marius (06:39) Yeah.
So, so, so, this is an op-ed piece in, the wall suit journal and it’s written by, or, you know, written by an AI, but it came as written from a Stanley, Druckenmiller, which is a really well respected, billionaire, hedge fund, Fund head manager, like definitely one of the
big smart people in the world of finance. So it’s coming from somebody with a lot of expertise and his expertise is certainly not in question here. Right. The thing that why it became so like contentious is that
The op-ed was clearly written by AI. Like it was not like may have been written by AI. It was clearly written by AI. So the conversation is like, where is the trust now? Right. It was written by somebody who has the expertise. as interviewed later, the author said like, look, I’ve been using spell check my entire life. The points in here were my points.
And I felt that the AI did a better way articulating my points than I could have had. So I see nothing wrong in putting that as my opinion piece. That was the author’s point of view, but many had the contrary point of view of saying, like, if you’re making conversation around policy and things that are really important in setting your point, should be your words, not AI’s words, because we cannot distinguish.
where your ideas started and where AI took over, right? So it created a huge controversy around trust. And there are definitely two camps. There’s one camp that says clearly what he was writing about was not AI stuff. This was very complex things that only he knows about it. And you could hear clearly hear that his thoughts were in there. And the other camp is we do not know where AI enhanced his ideas.
versus AI embellished and created its own point of view in the article.
Chris Heuer (08:40) Hmm.
Yeah. Well and and that providence really matters a lot of times when we’re trying to ascertain whether or not we should believe and act upon a given piece of advice. And you’re when you’re
Marius (08:49) Yeah.
Chris Heuer (08:49) dealing with financial information like this, that’s a particularly high cost area of making that decision.
Marius (08:55) Yeah. But
I want to add one other thing that I guess remember to his defense. He said that in the past, because he wrote many op-ed articles in the Wall Street Journal. It’s not the first time that he’s did that because again, he’s recognized figure over last 20 years. He said in the past, he had a PR firm write the article on his behalf
Chris Heuer (09:13) Mm.
Marius (09:14) and publish. And he asked, where is the difference?
Chris Heuer (09:17) Yeah. And and you know, I was just talking about with somebody, I I went to the CMX meetup here in San Francisco last night and was talking to another community manager about that. And you know, it’s the idea of the people who have money have the advantages of hiring other people for it. But the reality is today is that while AI isn’t perfect, it’s certainly gotten a lot better and it gets better every couple of weeks. it has reached the point of at least being conversationally intelligent so that we can not
discern whether it’s really a human or a AI that’s delivering it. So having having that perspective there is interesting. And so this is where I think this next bit is really important. Any professional, I believe today, if they’re going to make use of AI in their writing, needs to establish what I call their personal intelligence system. Whether it is a connected folder from your chosen AI platform that contains all of your writing
And you’re reading and the research PDFs and everything else that you’re relying on, or something like that, you need to be able to discern and define what is the corpus of knowledge that I rely on that I believe is true. And that if you’re relying on that when you’re using it for your writing, the chances of it being accurate are higher because it’s not going out to the whole universe of everything that’s out there trying to find the right things.
It’s pointed at the things you already believe are the right things. The other component to this is actually setting up the prompt, which is why one of the first articles we wrote for communityassistant.ai is the one that is about measuring twice, prompting once. How do we consider what we actually want to say?
And so it is about the writing process. And if that writing process is going to a PR firm and saying, hey, I believe this, go off through the research and write me an article, still having that back and forth with the PR firm. You’re not taking that from the PR firm and just letting it go. You’re looking at it, reviewing it, and making sure it represents your words, making sure it’s accurate.
Marius (11:05) Okay.
Chris Heuer (11:12) And hopefully ensuring that your main points are coming across. And sometimes in that process, whether it’s a PR firm or an AI, you’re gonna discover new facts or new points of view that you wanna incorporate. And I think it’s that iterative process that turns it from being potential AI slop with an easy button into something more meaningful and useful to the consumers of the content you’re producing.
Marius (11:34) Sure. But I think that the differentiator is now, and I’m curious to hear your opinion of it.
So then, I mean, for humans, we have the same thing. I would call like just paying, bring attention to politicians and their speeches, right? Human written,
Chris Heuer (11:47) Mm. Well
Marius (11:50) political, sometimes when I hear this, it’s like, wow, you managed to speak for 30 minutes and not say a single thing. It’s a talent that politicians have like always sort of mastered.
So I think then the definition of slop may not be that is simply just AI. It has to be taken also the content on it. And like you said, is it valid and can we trust it so that AI content can be actually valuable? Because humans can certainly create a lot of slop as presented in politician speeches.
Chris Heuer (12:22) So this really begs the question, Marius. is there a place where AI assisted writing will make a community better specifically?
Marius (12:30) Yeah.
So I think, I think yes. with parenthesis, because again, it’s going to be hard to define this slop piece, but I do believe that, because haven’t been in the industry for a long time. I know people that had always a difficult time expressing themselves, either being that they were not good writers or, their English is not their first language. Right. and they want to contribute in English. and they are.
subject matter experts, deep subject matter experts on the things that they are talking about. Right. So for those individuals having the ability to say something they want to say in badly written and spelled English, and I’m one of those people in many cases, and then clicking a button and then getting something back that I can quickly reread and go like, yeah, that’s what I wanted to say. And that is much better said than I could have put it into words.
I think that content is very valuable in the community. I also think there’s value from the moderator side and the company answering on some of those threads or support, take it like support,
Chris Heuer (13:37) Mm.
Marius (13:39) Assuming that the answer includes documentation, right?
Being able to quote something quickly, answer it, and then pointing it to evidence, which documentation would be the evidence, and quoting something from the documents directly and pointing them to the right answer quickly, AI can do far better than us and much quicker than us, and I think adds a ton of value. I think there
Chris Heuer (14:03) Yeah. So the
Marius (14:04) are some things to let us think. The things where it breaks apart is if the answer becomes
generic and you are not the expert in answering it. So if you just go in there and say, I think the answer is this, can you come up with what the right answer may be and make me look good? If that is your prompt, then I have an issue of like, why you being in the community and why you answering something where you are not the expert in. So that that’s, think where, where I think it’s can become sloppy. And that’s where I think LinkedIn is.
having a reaction to the content that’s being created where everybody’s a thought leader.
Chris Heuer (14:42) Hmm. Yeah, and again, it it matters on the context of of the space that we’re in here. one of the definitions of slop that I came across, but it was one the early definitions of this was talking about low effort, low quality. And that effort element gets to the process component of this all, but it’s also a matter of intent. And you’re getting to the intent now with that.
And I think that is really the question here. and I agree with you broadly on the use of AI for writing, and I’ve benefited from it, and I’ll come back and talk about that in a minute. But the idea that people are using it to misrepresent their expertise.
And you know, the integrity cost is more than trust.
It can actually be economic and financial directly to an organization here. but you know, there are places where individuals now are using it to reply to comments. There’s a comment to one of the video clips I shared from our last episode that I read, and I’m like, that looks like slop to me. And so here’s an individual trying to look better than they actually are and creating an answer. And I can no longer discern if that individual actually knows what they’re talking about or not. And therefore, I immediately degrade my perception of it.
And so,
Marius (15:46) Yeah.
Chris Heuer (15:47) you know, what about those situations here where the AI writing is making it look like somebody knows what they’re talking about, but they don’t? What is the real impact on that? Because again, we’re talking the process shouldn’t matter, but the outcomes should, at least that’s what I believe. And in this case here, if the outcome actually added to the conversation but decreased the trust of the individual who added it, what is the real outcome?
Marius (16:10) Yeah, so I believe that communities will somewhat self-police in this. What I think you will see happening is when somebody brings expertise that is not authentic and it doesn’t come out authentic from that individual.
somebody is going to go in the description or in the comments and say, this kind of looks like AI slop to me, like you just did. And one of the basis of like successful communities always has been that people are motivated by being recognized for the knowledge, right?
think that when they’re getting a negative feedback, such as like, this was too AI sloppy, they will start reconsidering how they engage in community next time. And it can have two effects. One, they never come back to the community because they felt put off. Fine. You provided.
AI slop anyways, what’s your value two they’re considering that this is like, yeah, maybe I should change how I’ve done next time. Maybe I can, I can rewrite what the AI said in order for me to be heard, right? Cause they want to be heard and then it had the positive effect of, of, of changing that. So I don’t believe that we are moving to a world where it’s going to be AI answering AI. It’s like kind of like an AI, a driven community where we just seeing AI going back and forth.
because the humans will just not stand for it. So I think there’s a self-correction that’s going to happen here. Now,
We can talk about creating terms and policies for the communities, which I think is another topic we can go deeper into and see if that’s needed. But I would hope that the communities and humans kind of like step up here.
Chris Heuer (17:51) Yeah, I well we’ll get we’ll get to policies a little more in a bit. I I I want to go a little bit deeper on this particular topic because, you know, I actually I’ve so personally I use AI a lot in my writing, but I use it as part of the process as a tool, not as an easy button to just produce an article for me and be done with it. I use it to help you know get concise with my thoughts. It turns out that I failed English in seventh and eighth grade
And I probably have a natural predilection to not being good at writing as it is,
Some people are just not good at writing. So if they can use a tool like a Grammarly or a spell check to make sure that they’re doing better, what’s wrong with that to refine their writing? And I think the answer to it is that it then makes it seem like they are better than they are. And I don’t know if you know, I don’t know
Marius (18:25) And. you
Chris Heuer (18:38) that that’s necessarily true, but I think we have a problem discerning intention and the real authenticity of of who an individual is. And we’ve already seen the trust erosion of whether or not
in on one side where people believe more in AI answers from Google, on the other side of it, I’m believing less and less of facts just as a result of the disinformation across the entire, you know, society right now.
So I need to now know provenance and and process and I need to know the individual’s intent and regular workflow in order to know whether or not it’s trustworthy. So it’s harder and harder to discern what to trust and who to trust as a result of this world. Does that matter or or is it the or is it the outcome? Yeah.
Marius (19:17) Okay.
well, let me ask you about this then. So if you were then to go to a community and you ask a very specific question and people start responding and you’re seeing something that is more.
AI driven, would that confusion lead you into less trusting communities and trying to find the sources somewhere else? Or you go to community and then validate the answers that you’re hearing from other humans. And how do you validate them? Do you go back to AI to validate it? Which I can see happening. I can see the whole loop. Like you’re going to a community, human says something, and then you go to AI to validate what the human said.
Chris Heuer (19:58) Well, I’ve certainly done that, and I’ve done that with what AI says because of in my experience I can recognize oftentimes that, like, that’s just wrong. And so because of having experience and and having been in a particular area or domain of of knowledge for a long time, I know the difference between right and wrong along a lot of things, but not everything. And so I’m able to go, that doesn’t seem right. Or I just have that feeling that doesn’t seem right, include multiple sources to check it. But again, the best one is direct experience.
Have I actually done this thing or tried this thing and what is my experience from it? But of course, what we’re trying to do in research is avoid wasting time and producing less than ideal outcomes. So we’re trying to get that answer and that process and that whole plan kind of worked out before we get to that point of actually experiencing a loss. But, anyways.
Marius (20:44) So what I’m hearing you,
I think what I’m hearing is you just need the evidence, right?
Chris Heuer (20:49) Yeah.
Marius (20:50) You want to hear stuff, but you want to have the evidence delivered with it. And I think that that is something that we do bring it back to community, right? In tools like AI search, right? There’s one thing to go to a community or other publications and put something in search and you get an answer.
There’s another one to get actually the evidence and read the evidence where the answer is coming from. So in terms of communities, right, you would get an answer and then you have like the threads below it, like here’s where the answer is coming from. So you could then click on the answer and hopefully that one is not AI slop. And then you go like, okay, I understand the evidence and the answer now.
make sense. So it’s really kind of like, where does the evidence lie? And the evidence could be a research paper, it could be a news article, who knows what it is, but it has to be, the evidence has to be more clear, right?
Chris Heuer (21:43) Yeah. And in fact, I think Jennifer McClure in one of our Team Flow Institute research reports really stated this rather clearly in here is that you have to show your work. And it’s showing your work that allows people to trust it more in terms of the providence.
But I don’t think for an article, many times people are going, Okay, here’s a link to the AI prompt I use to write this article or to write this answer.
Marius (22:05) No, and we
wouldn’t do it like again, coming in this example of the Wall Street Journal guy, right? mean, in the past, he would have like a 15 minutes conversation with somebody from a PR firm, discuss what he wants to say. And then the journalist or
PR person with journalistic backgrounds would have taken this, written the report, and then he would publish in his name. That was never a writing on like, hey, this was based on a 15 conversation. Here’s the transcript I did with the individual. Like you wouldn’t have that. And the conversation, the 15 minutes, you know, technically was the prompt, right?
Chris Heuer (22:44) Yeah. Well well, that’s really interesting because you’re getting into the point of the distillation, right? The the
Marius (22:50) Mm-hmm.
Chris Heuer (22:50) the TLDR, the too long didn’t read. How can we summarize it in such a way and how can we end up with trust in the summary? That’s a wholly different issue. and it gets into another problem that I referenced earlier in my bow slop measurement kind of model, which is bloated. and one of the biggest problems I’ve had over the last year, in particular working with different people, is when they send me
you know, this wall of text, 8,000 words, to try to explain something to me instead of bothering to have a 15 minute conversation. And that puts all the burden on me as a consumer. And then me trying to figure out what me and my judgment of what’s important in here.
And so how do we actually get this point that it is both easier for the creator and easier for the consumer? Sometimes it doesn’t require sending 8,000 words, it requires a fifteen minute conversation.
So we really need to apply our judgment to the process as much as the outcome as we are dealing with this. but that’s actually again, I think the real debate underneath it all. And you hit on this, but I want to come back to it one more time here. How do you get to better outcomes in a community with AI? Where is AI actually best deployed in a community right now?
Chris Heuer (24:00) Hey, sorry to interrupt, but I need to thank our sponsor, Higher Logic. This episode and all episodes of this season one is supported by Higher Logic. The Vanilla platform is where customer communities live. Discussions, QA, knowledge base, ideation, moderation, all in one place and connected to the support and CRM tools your team already runs on. Higher Logic is community first. It’s not a feature bolted onto something else. It’s a whole product built by people who have been doing this for more than fifteen years. Here’s why that matters on this show.
When AI answers the question, vanilla is where the humans show up next, where members find each other and where trust actually gets built. Visit them at higherlogic.com and tell them that you saw them on communityassistant.ai. Now back to the show.
Chris Heuer (24:47) but I want to really distill it down to something important here is that it seems to me that the three things that you’ve been talking about AI doing for community really comes down to accessibility. Can AI
Marius (24:59) Great.
Chris Heuer (25:00) access your data, your community comments, your documents, your knowledge bases?
Marius (25:04) you
Chris Heuer (25:04) support forums, support tickets, even if you can do that. That’s awesome. The enablement,
can we get members the answers? Can we actually shape the experience better by understanding what it is people are trying to accomplish and analyzing patterns and data to enable people to produce the outcomes that they’re trying to get? And then broadly is the discoverability. How do we actually, and this gets to the the geo sort of work, how do we enable the
Marius (25:27) Mm-hmm. Okay.
Chris Heuer (25:30) bright content from bright people in our community?
be discovered by people outside of it, to either make a decision to go with our given product or service or to join our association and be a part of the professional community? Or is it just to know that exists or get an answer outside of the community even and to be
Marius (25:45) Yep.
Chris Heuer (25:46) able to position the community as a place of authority so that other people think of the reputation of it more highly? Does that kind of reflect it, is there anything missing in that?
Marius (25:53) Yeah, doesn’t
know I think this is exactly right. The only thing I would point out when it comes to accessibility and it comes again and like how we’re thinking around community. It’s the easy one to think about community like all those other knowledge areas that can benefit me as a community manager to or inform the community members to get the most out of the community. But the opposite is true too. So it’s not a one way street.
The support agent, for example, if it has the community knowledge, when they have to answer a question and they are using AI to help a customer, when that AI has access to the community, bringing in answers from the community could be hugely valuable for the customer support rep in answering that particular question. The product development.
lead, you know, when they are building a new tool and they’re leveraging now AI in how to build the tool and how to think about the persona of the users and what the user needs. Having the context from the community of what the user needs are will be hugely valuable to developing the next product, because as a real humans giving real input and it’s not something, you know, driven from AI. So the accessibility is.
bi-directional, just want to make sure that
Chris Heuer (27:15) Mm.
Marius (27:16) that is understood by community managers. And it’s really the value that we are representing because we always said there’s this tremendous value of the customers that can be unlocked through communities. This is now a real way of like how it’s actually going to go across the organizations to all organizations, including like sales teams when they are going to talk about certain
features, know, the AI will tell them like, Hey, the community does not like it when you talk it this way. The community really responds well when you showing this value propositions. now, now they can test some of them at marketing, right? I could go on and on and on.
Chris Heuer (27:57) Yeah.
Marius (27:57) it’s this bi-directional piece and, and bring it back to the, the, the, the slop thing. If the content is.
valuable and the member express themselves through a thoughtful AI written content piece, it actually going to be valuable because of this bi-directional accessibility, because it’s going to be another AI reading this and it’s going to get real a lot of insight from the level of detail AI sometimes goes into. As long as, like you said, it is the right thing, it’s not wrong, as long as it’s all true.
and factual and great. And the opposite will be true as well. If it’s bad,
Chris Heuer (28:37) Yeah.
Marius (28:38) it’s going to proliferate everywhere. So imagine like AI gives the wrong answer. Support guys is reading an AI thing that is pulling the wrong eyes and doing the wrong citation. And it’s going to kind of like now bring it forward to another customer, this wrong information.
Chris Heuer (28:52) Well and and that well there there gets to be a a more important baseline issue here, which is actually the use of AI at all inside community. Because I I’ve been talking to a few community managers who will really
Marius (29:01) Yeah.
Chris Heuer (29:02) say, I am not going to use AI in my community. That is not the point here. We are here for c human connection and to connect individual users with them or individual members of my community with each other, and for us to connect with them as a company or as an organization to do this, and therefore
All AI is bad.
And so I I think this is still being determined. And there are many communities out there who will not use it at all and who are just simply anti-AI.
Marius (29:26) Absolutely.
Chris Heuer (29:28) What what is their approach? And what are those people who are against AI gonna do?
Marius (29:33) It’s going to be a good conversation. I come across that all the time too. And I go both ways. It’s a pendulum, right? The other people that go there and I go like, if you do that, you’re taking a huge value proposition off the community. Is it more important that we have humans giving answers or is it more important that the member comes out with value from the community? Right? So that’s one side of the story.
But I’ve seen the opposite too. You know, I’ve been to a community workshop, to, but I’ve attended one and the question was like, what is your ideal new tool? And a community manager stood up and said, I want a bot that answers all questions in the community. And then I was like, then what do you have the community for if you have a bot answering all questions? So I don’t want either. I need to have
Chris Heuer (30:20) Mm.
Marius (30:20) a hybrid approach.
where the AI is a system humans. One of the things that my boss came up with, which I was really cool, when we built the AI search piece, we thought about, if the answer lives in the community, let’s bring it out. But if it doesn’t, what AI says is, I don’t have a good answer. I recommend you asking the question in the community. Now we are kind of creating the right
quality in the community because we’re bringing new questions that require new answers. That’s what we want in the community. So we enriching it and we are helping the members also if the answer already has been given multiple times, putting it in the community doesn’t add a ton of value, right? Because it’s been answered many times before. Why go through that whole process of putting the question in and somebody is going to come in there. like, Hey, this answer is in this thread.
And then maybe somebody even puts something in that’s not right. Have AI give the right answer and move that member out of the community because they came in with a specific goal, they achieved their goal and that has value in my opinion. So I think it has to be a hybrid. It’s not one or the other. I definitely don’t want to have a community only driven by AI. That would be bad.
Chris Heuer (31:41) Yeah.
Well and and within this as well, we we have an another kind of issue in here, and we we talk about this briefly with this comment that I got on my LinkedIn thing. And you know, LinkedIn added the button seems like AI slot, so people can start reporting it, and I that accrues to someone’s reputation over time. but my issue was in regards to calling it out as like this seems like AI AI produced answer. Is this really what you think and are you there? And I think that it really
You know, the idea of a community self-policing on these things is interesting. but I think the different values that people have and the the the potential for conflict within the community that could cause bigger challenges, creates a real risk. And and again, I don’t know we have all the answers yet. I think we need more people experimenting with this to start producing more case studies.
Marius (32:28) I can give an example of what I think. Community managers, please don’t do that. But I think it would be a fun experiment to do. We all know leader boards, right? You post something, you get points, and then you can go, who’s the more active person? Imagine we had this button, AI Slop, and we have a board of the leaders that get the most AI
Chris Heuer (32:48) Hmm.
Marius (32:48) Slop clicks.
It’s a little bit shaming those guys, so that’s why I say not do it. But what effect would that have? I’m pretty sure you don’t want to see your name at the top of the list of being called out AI slop the most.
Chris Heuer (33:03) Yeah.
Marius (33:03) That would be a really quick way of removing all AI slop.
Chris Heuer (33:08) Yeah, well, there’s an interesting way you might be able to flip that and do that, which is actually showing the positive, which is the percentages of posts or comments or engagements that were not ranked that way, right? So we can
Marius (33:20) Yeah.
Chris Heuer (33:20) show that they got 98% likely human responses versus
Marius (33:24) Yep.
Chris Heuer (33:25) 15% AI slot. So if we just inverse what we focus on, we can achieve potentially the same impact because if you’re down that list for not being human.
trusted answers then then that’s a way to produce that sort of shame without focusing on the negative.
Marius (33:41) Absolutely.
those are the things we think we need one now to think about as community managers in this age of AI. How can we do these type of experiments where we allowing both, but in the end it has to have value. It all comes down
Chris Heuer (33:56) Yeah, well we’re we’re we’re
Marius (33:57) to the value piece, right?
Chris Heuer (33:59) Well, and so we’re getting here near the end. So this is a good time to shift and actually a good place to shift off on, which is what should the community manager be doing on on Monday? And of course, we we know some elements of this is automated moderation. And in the work map report that I produced, one of those key elements is that automated moderation to do that Pareto principle sort of eighty twenty past so that I don’t have to review every single comment, but I can look at the gray area ones. I think that’s really important.
But there’s another issue here that I think is more worthy of discussion that that’s a little bit more challenging. And it’s getting into these ideas of leaderboards and things like that is what is your AI policy as a community manager? And so are you seeing individuals in your
Higher Logic customer base or elsewhere in the world where community managers are developing an AI policy specifically.
Marius (34:49) So I don’t, I, would expect to see it more, but you currently don’t. I think that the next large, like community events will start having those conversations as AI content will be more more prevalent inside communities. And they’re going to be, you know, there’s two pieces to this too, that I want to make. There’s the policy around AI bots coming into your,
content and, and disguising as humans and providing
Chris Heuer (35:17) Mm.
Marius (35:17) answers. So you want to have like a clear policy on that. And you want to start thinking about what is the impact of AI bots in the community. And then secondary is like the use of AI in the community. I think that if I were Monday and we’re going like into like, you know, what a community manager should do on Monday, as we set as community managers and,
talked about what is acceptable and what is not acceptable in community. What are the rules about removing a post? Is it authenticity? Is it language? Is it that they are talking about competitive products? On the association side, is it that they are bad mouthing the directors or the association overall? We don’t want that type of content. I think it’s worth having that conversation.
with your team and with your executive and saying, what’s the expectation when somebody comes to the community when this happens? Is that a good thing or a bad thing? And then being transparent and maybe even running your first thoughts by the community before you writing the new terms and saying, we believe a healthy community looks like this, what do you guys think? So then you setting and you’re creating those policies with the community together and then eventually write it down. So that when the situation
come up, you don’t, you’re not reactive on like, what shall we do now? You already anticipated it and you have a clear policy on, you know, removing that bot or removing that, slop content with a nice message, right? I can see easily saying like, Hey, you know, we remove this because it was sloppy, clearly written by AI, but we really value your opinion. wouldn’t you mind re rewriting it and reposting it and we will approve it. Right.
And this could be an automated message, right? What I just said. It doesn’t have to be every single time. can do it like every time you’re picking this up, you copy paste that in, close that thread and send that message to the user. And now you build some efficiency in it. So I don’t think it’s anything new. I think that it’s just that the conversations has not happened yet. And I think more people should be talking about it. And I don’t know, maybe it’s a cool site workshop on at some point of like what this policy would look like.
Chris Heuer (37:24) Yeah, I think that this is probably the one thing I would encourage, you know, community managers to do is to kind of at least get an understanding of what they want. And I would definitely encourage them to ask the community like that.
Marius (37:36) Your job is really to enable what communities need in order to be successful. mean, community managers are enablers.
Chris Heuer (37:47) Yeah. And and in that way, the AI is supposed to be the enabler for the community manager.
again, AI slop is such a broad area because it touches on societal views of it. What is the long-term consequences of its production? I mean, we didn’t even get into the fact that that overgeneration or that excessive generation of content and then the use of the AI to reference AI generated content reduces the quality of it technically and all those other issues around it. But I I think really the the real issue here is a matter of trust.
a matter of accuracy and a matter of the expectations that your community has on what is the purpose. And you know, your question on using AI to answer questions, I I think again, if somebody’s typing in a search bar, you know, leveraging AI is no different than using, you know, search technology to produce it. In fact, you could argue search technology is a form of artificial intelligence. But if somebody’s posting a comment into a community forum expecting an answer from another human.
Marius (38:41) So
Chris Heuer (38:50) having a a bot just reply automatically, I I don’t know. I think we’re gonna have to find out over time if people are okay with that. but generally speaking, I I would think that it would be better for a human to reply and
Marius (38:55) So
Chris Heuer (39:03) say, well, here is what the knowledge base says, and here’s a pointer for you to go look at. And just even having that one sentence introduction to an AI answer can change the shape of it. And that’s why, you know, our central thesis here is the idea that.
it enabled us to spend more time doing the human work of relating. And I think even in that small way of saying, well, here’s an AI answer to it, that adds a human element to it. I I don’t know. Do you agree with that or is that still just another form of an easy button?
Marius (39:20) No. No.
No, no, no. I 100 % agree. think that that is sort of like where, but I think that what you’re saying will become more more natural. I think this is just going to happen over time. It’s just this right now is very new and the pendulum is either all the way on one end or all the way the other end. Like people need to come to exactly what you just said. It’s going to be sort of this mixture of like, here’s what I think. And by the way, I pulled this AI answer to back up what I just said. Feel free to read through it if you want to have all the details, but let me shorten it up for you.
you know, then it becomes human, but it is AI backed, which is, think is going to be much more interesting than it’s going to be AI written with a little bit of human backing. I love it the other way around. I think
Chris Heuer (40:15) Yeah. Yeah.
Marius (40:17) we’re going to get there naturally. I think
Chris Heuer (40:19) Agree.
Marius (40:19) we will see that transition as people are still need to want to be recognized, still want to be transparent because authenticity is still king. And I think people will on their own come up with that.
Chris Heuer (40:31) So so something’s really hitting me here at a at a high level, which is, you know, the purpose of community. And the as I watch communities evolve, particularly organization-based, whether it’s a nonprofit, for profit association, brand, SaaS platform, whatever it is, that are people coming to the community to get answers to questions? Are they coming to community for connection? So that’s a fundamental one. And this leads me to the question of.
Should people really even come to community for questions if an AI can provide a better answer? And I I don’t know, where do you where do you land on that? I mean, I guess I I don’t know where do you land on that? Because I think
it could be said that they don’t need to rely on community for answers anymore because the AI can provide it, but is it still the right answer?
Marius (41:16) No, it’s,
it’s, but, keep in mind that AI only knows all the stuff because it’s read what it currently knows. So it is only at as knowledgeable as a certain piece in time. And as new content gets raised by humans, it needs to catch up to the new content.
some of the answers are uniquely human and highly specialized because they are built in experiences.
that are too unique for AI really to pick up. So it is about me as a user is experiencing this. I have a question to somebody that may have had the same experience, right? So it’s not generic
Chris Heuer (41:58) Mm.
Marius (41:59) questions. It’s not like, how do I fix my type, my thing, right? That probably is documented somewhere AI can figure.
this all out. It’s more, I’m in this situation. I have this opinion. Do you agree with me or not? I want to have different point of views. And where this conversation coming, I cannot replicate this different point of views. There’s lot of nuance, I think, to community. And it is beyond connection.
Chris Heuer (42:28) Well, and this comes back to what we’re still recommending people do, which is go and create community managers do, excuse me, is to go and create an AI policy. And I guess the point here being is that you really do need to revisit what is your mission and purpose for your community
Marius (42:41) Yes.
Chris Heuer (42:42) and understand what that is and make sure that’s up to date as well as you’re starting to write these things. And then again, your values. What are your values that you have in here and and how do you approach AI and being able to express that to everyone? Certainly people are going to disagree.
But you should take your stand and make it public and make it visible and available to people. And if they want to have a debate about it, that’s fine. But you need to take your stand where you need to be on this thing. Marius, we could we could keep going and have another couple of hours on this, but we don’t have that time.
Marius (43:08) Yep, for sure.
Chris Heuer (43:11) and so we really need to wrap up and so much more to get into. And as I said, you know, one of the things that I’m really looking forward to is understanding.
gosh, where are people actually deploying AI in their communities? So what is the point of your community? Seems to be the first question that we need to answer. And what is the value we’re getting from it? And then how do we use AI?
To enable more of that and how do we actually refocus on what are the human components of that community? And fitting that all together, I think, is really important. But anyways, a lot more to talk about here. And that’s what the next several episodes are going to be, is to go deeper and deeper into how community managers are deploying AI to make their jobs
Marius (43:34) Okay.
Chris Heuer (43:50) easier to manage and to help them do more of the human work and to enable their organizations to accomplish the goals that they have for their communities.
And to ensure that we’re actually measuring those appropriately. any other closing thoughts that you think is very important that we want to highlight or that you perhaps didn’t get to state earlier?
Marius (44:07) No, I think we covered it all. Just really emphasizing, take the time and go to the whiteboard with your team, with your executive, with your manager.
And go through what if scenarios and design the community that you want to have and the behaviors you want to have in the community. then create those expectations of your members and communicate to your members, like what the community should be and what you want it to be. And I think that’s where you’re going to get responses back and they would follow your direction. You just have to just set a little bit like what the garden should look like.
Chris Heuer (44:42) Yeah, and that follows within the reasons for creating the work map report is to not just go ahead and enable a technology to do something for your community or for yourself because the technology can do it, but actually intentionally redesigning your workflows, thinking about what are the components we can offload to technology versus the ones that are uniquely human that we should retain and manage ourselves and what the value is of those different choices within it. So this is
Chris Heuer with my co-host Marius Ciortea for CommunityAssistant.ai, helping you determine which parts of the work are uniquely human that you can do, and which parts of the work we can hand over to the AI and give it away to your assistant to use AI as a tool. And fascinating conversation and so much more to discuss and learn. And that’s why we’re trying to learn it all together. and really I have to again thank our sponsor, HigherLogic, and
really just wonderful in terms of being able to support this conversation and talk about the broad issues instead about just the specific technical implementation and unique issues that come up with vanilla and the platform and what higher logic is actually doing in the world to look at the broader implications for community managers and for the industry as a whole. I’m just really grateful that we’re able to have the time put into this.
Marius (45:38) So
Chris Heuer (45:56) And Marius, I really just thank your vision for investing in this and and joining me in sharing what you know and making this about the industry more so than just about
So thank you so very much. And for those of you out there, I look forward to seeing you more in the future, hearing more of your feedback. Please, if you want to submit yourself as a guest, we’re definitely going to have guests on future episodes and have other segments that we’ll introduce. What I really want to know
Marius (46:08) Thank you, thank you. Okay.
Chris Heuer (46:23) is your feedback on that work map report. So please go ahead and give a look at that. I’ll give a link and I’ll put a little bit more about the moderation component in particular because I think that’s a real low-hanging fruit that community managers should do.
And we’ll continue to evolve
Marius (46:36) you
Chris Heuer (46:37) that conversation and learn together. And that’s the real goal that we have here. We thank you very much for joining us, and we’ll see you here next week.
Thank you very much, Marius. Have a great day.
Marius (46:47) Thank you.
Made with Claude, not by it · CC BY-NC 4.0