New episode is out! @dsearls and @katherined talk to @eze_lanza and Tony Mongkolsmai about #ChatGPT, generative #AI, and #opensource software.

Episode 135 - Experts Weigh in on ChatGPT. Listen here: https://www.reality2cast.com/135
#podcast #newEpisode #tech

@reality2cast

Hi, thanks for an interesting discussion.

Here is one question I think you missed: for that kind of AI system that is "trained" on data from the internet, suppose now that in the future the "new" content added to web pages will be increasingly generated by such systems. Then when the systems are to be improved by training on the new added content, they are increasingly taking in their own earlier output as new training input.

1/?

@dsearls @katherined @eze_lanza

@reality2cast

Will the systems still improve in that situation, or will things stagnate because of lack of real input from real human beings?

I guess that if you think there is no problem with that, then you could already construct such a feedback loop where the system eats its own output as input. Does that work, or not?

Will "original content" created by actual humans capable of original thought become increasingly rare and increasingly valuable?

2/2

@dsearls @katherined @eze_lanza

@eliasr @reality2cast @dsearls @katherined Hey good point! I see it as when used in scenarios like computer vision, where you can create synthetic data to increase your dataset size (data augmentation). It's also useful when you have a very little amount of data to train your model, it can help on the training task when you are in this situation and it works well. An important disclaimer here is that the models trained with synthetic data has lower accuracy than those trained on real data 1/2