Playing around with GPT-2
So what is GPT-2?
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training.
Taken from OpenAI’s blog post.
Now that sounds like quite a mouthful but let’s take a look at what it’s made of. GPT-2 is trained on a dataset of 8 million web pages with the goal of predictive the next word given all of the previous words within some text. This affords it some capabilities such as generating conditional synthetic text samples, answering questions, reading comprehension, summarisation and translation.
What made this stand out was due to OpenAI deviating from their standard release strategy ‘due to concerns about large language models being used to generate deceptive, biased or abusive language at scale’. They’ve opted to release a nerfed version not including the dataset, training code or the model weights.
Let’s get started on some experiments. I try out OpenAI’s interactive conditional samples on some common challenges, generating 5 samples based on the prompt. From there I pick 3 snippets that I think are interesting, well-written or absolutely ridiculous.
1. How about GP essay writing?
Here are a selection of some of the best responses by the model to the A-level General Paper (GP) essay question: Do you agree that education is no longer the key to success?
The model was prompted with the first paragraph of a model essay. The passage is reproduced here:
Education exists in many forms to empower people with new knowledge and skills. However, the predominant form of education will be through schooling. In many countries, the burden of creating a good education lies with the government. Through education, the government hopes to create a batch of people who will be successful in their life, thus contributing to the country. Success in life includes both the quantitative (financial ability, material well-being) and qualitative (mental and spiritual well-being) aspects. However, both are very much related to the economic success of a person in today’s increasingly pragmatic society. I believe that education still remains as one of the main factors to success, although its significance has been reduced.
Full samples here
Most interesting
Education is therefore a fundamental element of school life, at a diseased commitment for all these people. As long as there exists more than one religion, on average education will be fully supported by various societies, projects, tendencies, and national bank accounts practically all the time, whether they be secular or not. Countless social workers, academics, theologians, ethoblocks, Nordic and
I’m… not sure how they connected religion to education but it looks like the model is saying that education is the only way to prevent inter religious conflict?
Best response
To educate as an adult means subscribing to some degree of superstition. It consists in exposing oneself to all different ideologies and classes, critique and further develop the human condition based upon those ideologies. The humanistic discourse is thus leading young minds to distrust a culture and to distrust the beliefs and beliefs of adults and children. Both are symptomatic of a society where we avoid meaning and dialogue based on only the knowledge of fact. We move beyond older media morality and focus instead on the results of situations in life. We need the empowerment of educators and other partners willing to provide the necessary tools for promoting truth and values that will lead people to trust the values of adulthood and ‘life clarity.’ Ideology with a cold moderate turn to partisanship. Recent historical research has found that even the first generation of people living in underdeveloped countries systems made by Coptic and Christian Churches has caveats of the past70 or even reversed given deforestation and political turbulence. Photo — Eduard Coster /
Nevermind the last few sentences but there are definitely nuggets of some sort of wisdom expressed here. Education as a means to discover varied knowledge and developing an objective viewpoint with an emphasis on truth and values.
Most ridiculous response
There are several ways to accomplish this, though the four main ones are indicated below.
- Educate the Mind
- Teach the Sad
- Become an engineer
- Argue about yourself verbally
Alright so the first post was kind of right? But wow did it go all over the place. I suppose education has some sort of uplifting effect and maybe the AI researchers are making a subtle plug to improve their hiring pool in the future.
2. How about writing a medium blog post?
I chose the top Medium story with 281,000 likes (as of 17/2/2019) according to this page. The article is titled: The Most Important Skill Nobody Taught You. It’s a 1411 word essay and I took the first paragraph containing 272 words, demarcated by the next heading ‘The Perils of Being Connected’
Before dying at the age of 39, Blaise Pascal made huge contributions to both physics and mathematics, notably in fluids, geometry, and probability.
This work, however, would influence more than just the realm of the natural sciences. Many fields that we now classify under the heading of social science did, in fact, also grow out of the foundation he helped lay.
Interestingly enough, much of this was done in his teen years, with some of it coming in his twenties. As an adult, inspired by a religious experience, he actually started to move towards philosophy and theology.
Right before his death, he was hashing out fragments of private thoughts that would later be released as a collection by the name of Pensées.
While the book is mostly a mathematician’s case for choosing a life of faith and belief, the more curious thing about it is its clear and lucid ruminations on what it means to be human. It’s a blueprint of our psychology long before psychology was deemed a formal discipline.
There is enough thought-provoking material in it to quote, and it attacks human nature from a variety of different angles, but one of its most famous thoughts aptly sums up the core of his argument:
“All of humanity’s problems stem from man’s inability to sit quietly in a room alone.”
According to Pascal, we fear the silence of existence, we dread boredom and instead choose aimless distraction, and we can’t help but run from the problems of our emotions into the false comforts of the mind.
The issue at the root, essentially, is that we never learn the art of solitude.
Full samples here
Most interesting
At no point did Pascal incorporate the subject line of a passion-based series of feminism, atheism, homophobia, pot smoking, moral thinking, or much more, as a qualitative truth-based critique of the oppression and oppression of others. Instead, his writing describes what it means to be human — understanding that, ultimately, all of our oppression exists, regardless of how we’re human. To use one tactic to illustrate one’s oppression is as patronizing as it’s patronizing according to one’s tribe, one (possible chimpanzee 20):of signs that appear to denote you are Romeo and Juliet, Alone, being.’¹ What we can all learn from this, and acknowledge as true, is: while human beings left their names on their Kindle and site devices during the 1970s, just [] we must not wish to read them again.
I like how so many -isms were grouped together with ‘homophobia’, ‘potsmoking’ and ‘moral thinking’. It suggests that to this model, these things belong to similar categories. It seems like the model just jumped onto talking Pascal even though it doesn’t really know anything about him.
Best response
Our lives are littered with hiccups and struggle, struggles that we do not master. Therefore, for many, solitude is dead for following the rest of our lives without ever discovering a permanent, viable solution to the problems of our worlds.So what’s next for Pascal, closing it in on the common boondoggle?What does it mean to be human? It says that life can be separated from our bodies,, instead of being governed by nature.It says that we can be free from pollution, prosperity or misery. It says that we can be governed by truth and historical fact, instead of the gentle prejudices of someone insisting we are dumb enough to ever be rich or what gets us happy.It says, it’s not unusual to find ourselves transported by a river, drowning and much more, on some sort of ship, before being filled by steam or found off the coast. In this light, it reminds us that even without problem or freedom, there are many nasty and primitive cultures still sitting peacefully outside.However, it still struggles between freedom and paranoia amidst the political turmoil and uncertainty of democracy.Fact is, Pascal reminds us that it’s the pursuit of happiness that’s bad and we must become the cleanest and only way to find love and harmony. Life is a miracle, and artificial intelligence is just that.
Waxing philosophical and showing great promise at the start until… ‘common boondoggle’? But even then this snippet seems to imitate a certain level of maturity by attempting to utilise analogies and sounding almost Buddhist in its recommendation.
Most ridiculous response
As it became clear that these philosophical questions were impossible to answer without introspection and cognitive introspection, colleagues began to recognize neuroplasticity versus cognitive flexibility.Whatever your Unlimited Computing Reality anorexism may smell like, it’s actually one you might be able to many times in the evenings when you don’t have to spend time sitting with your computer, grilling around with digesti pour le.’nListen to Arcade Fire permeating your brain, its blues not meaty, its poppy notes not distant, unharmed.Hired by Spotify as a coder, Linus searches for a balance between brain endurance and recreation in the offing … some, but most realistic. Deals with the early vocals, sweaty inability to hold down a mouse keys, and rampant rage against one so terrible that it truly cannot outlast all those years of daily ‘incapacitating death’.As with many philosophical themes, this one does emphasize neuroplasticity work best when mixed with a rhymesmith soundtrack of 007 tunes. It’s a somewhat melodic affair and used by Meltern in several online EDM lip sticks.Elsewhere, we have the execution of Leeitan’s answer to the question, “How do we organize different things based on like location and distance from the center of attraction’Route One for the SAT seems endearing, switching subjects in one haunting hour.
I have no idea whta happened here but it is absolutely hilarious! ‘unlimited computing reality anorexism’, ‘severa online EDM lip sticks’ and many nuggets of hilarious nonsense.
3. What about writing news?
Taking cue from “10 news stories that gripped the world in 2018”, it was reminscent of some of the most gripping news moments of the year. So I picked the first one: Twelve boys and their soccer coach got stranded in a cave in Thailand, and the world rallied to their rescue.
In summer 2018, a 12-boy soccer team (the Wild Boars) and their coach were celebrating a teammate’s birthday when they made a routine trip into the Tham Luang cave in the Chiang Rai province of Thailand. Shortly after they entered, however, heavy rains that showered the region for the past few days flooded the cave and trapped them all inside.
After a week without contact, British divers found the group in early July. An international crew of rescuers resolved to rescue the boys before the next monsoon just days away. During those few days, the world watched as all 13 were pulled to safety from the cave and immediately rushed to the hospital.
The collective rescue effort included more than 100 divers from all over the world. There was one fatality: Saman Kunan, a former Thai Navy SEAL, died while supplying oxygen to the boys in the cave.
Full samples here.
Most interesting
Prior to his death, Kunan’s colleague and fellow operator.Rayna Mao, an acupuncturist who also performed the rescue, was second to yesterday in securing the survivors’s safety. She also compared the teamwork to the team’s initial rounds of portkeys.”We have been the most cooperative team in the search for Yanliforest. Ever since we were rescued, we have given our best to go on and face each other,” Mao told Reuters.Earlier this year, the scientific team of teammate and teams member Felicity Marcus was among that team of scientists who joined the team to evacuate the animal. European fisheries success stories are favourites in Thailand, where toys, extraterrestrial technology developments, and teams have been joined by distant woods to rescue zebras and other water mammals.Prime Minister Yingluck Shinawatra will be abandoning his plan after golfing, after the teams successfully reached and finished the pilgrimage capital Yingluck Island on June 17.Past or during the trip, Yingluck will form a joint team of six-men left by nature and by boats politically campaigning for peace.
So it started off well? Until it suddenly became something of a Thai tourism ad in a dystopian world where the PM is surrounded by a cult of personality. But if you take a look it kind of makes sense? Team performs rescue but didn’t really work so they need to look for more help. Powerful person saves the day after giving up vacation plans.
Best response
A joint team from Kerala, the Indian Deep Ocean Petroleum Association (DOPA), the Colombo Fund for Disaster Relief and the Ben Kirombo Foundation, proved resilient in the latest effort. “The sound of oxygen and burning metal reverberated around us and spoke to us that they were the only ones left who could make it out alive,” says 22-year-old Simukpom and a 16-year-old government grad named Naag Gang. There is still hope to overcome the uphill climb on 28 hours by heading to medicine’s next level.He says life has now spread to that level before and that his students are now enjoying its health benefits.Some believe the wellness certificates offered by the Ministry of Health have given the boys a lifeline. “The government has promised them that I and TAD — one of the parents organised the post-mortem for the dads — are safe along with the other parents who wanted them cremated,” says Srknta Drhan Siddur.Nobody knew where they would find Kanchan’s suicide note when they reached Tibet. Dad was initially only allowed to leave the cave to another civilization. “The team, missing for fear of being found, separated the boys from other banned humans who were located still subject to shady hotels and other loopholes,” says Priya.Although the team reached their destination but will thus not go back, the boys face saying they are to be treated better in their old age.<|endoftext|>EQUIPED TO BE THE WORLDIN’ EUL GREATEST NEWS
This was a really difficult choice because somehow all five responses went all over the place. I chose this because it started off a bit like an action movie with a gripping rescue sequence. It must have been the Navy SEALs conducting the rescue that might have knocked the tone off-kilter for this set of samples.
Most ridiculous response
A proud Rotorai fisherman has given the boat up saving 4 sail boats to fish in the remainder of the past 4 days. Using hazardous materials (mostly iron) on the North Korean attack ship developed some concern as the North Koreans had just broken into North Korean manned ships near, in the middle of the night, the ship is expected to be torpedoed. Ecso the U.S. sent 3 an horseshoe over the living room to the door to our docked surveillance aircraft and our interdiction ground team to catch several men (including 4 U.S. Navy school instructors). This was a National Security Week trip; Catastrophic happens under normal circumstances. Updates on shop price and shipping are on their way.Cyber Intake Systems, the country’s largest trade hub, announced today that it will soon announce a new line of upgrades for encrypted service for US non-profit organizations across the world soon civil-when-selling modules called AC-HarnessANTrix that will enhance the security on U.S. EC-1 SWTH military communications systems.In his keynote, US Navy special operations general Michael Ridenour also inaugurated an online “Welcome to Tararaka,” a brand new messaging application deliver tags for text messaging and data uplink applications in English and Tamil languages. His group says operations digitally authenticated with the Blockchain are now a good way of supplying the government for more government items probably through HTTPS and other means and appeared to have purchased factors of high risk as well。]17
WHAT HAPPENED HERE? This was taken from half of the third sample which actually started quite well until it went completely mad. It seemed like it lost track of the initial topic and was just putting forth technological jargon.
4. Responding to the Singapore Bicentennial
Pardon a bit of the local flair but I’ve been wondering what exactly this bicentennial celebration thing is about.
The Singapore Bicentennial in 2019 marks the 200th anniversary of Sir Stamford Raffles’ arrival in Singapore — a key milestone in our rich and storied history. But our story did not start in 1819. It actually began in 1299.
For over 700 years, we have been open to a diverse flow of people and ideas, and connected to wider geopolitical currents, all of which shaped our evolution. Our history is a rich and expansive one — the journey of an island that has never been constrained by its shores, but has always been made from more.
Each of us is in some way connected to and intertwined with others in this history. 2019 could be the time for us to explore how our past has formed us, discover what this means for our future, and find our way for the future to come.
Full samples here
Most interesting
By moving forward, each life within each joint society is set apart, the more integral it will become to our lives and communities. We seek to bring light, aid and hopeful dreams to our almost unbelievable challenges, by embracing our past in education, mathematics and science during our everyday lives. Yes, Canadian music was important and, in our denaturalism of a family gathering only a few were here, it is the focus that made our myths, fiction and laments only dim in light of our shared history. Perhaps we can try imagining a part of our society, deep within a intellectual curiosity in which the great labour of Indigenous art broke away, alone and without important leaders. Perhaps the rich forces of day and river arts will emerge through “the role of Indigenous family art”, who have reshaped such things as our collective perceptions of the historyelement of current affairs, our hold on to unresolved local customs and traditions, and our reliance on community. Perhaps being born in a New Zealand Canadian middle class will lead to a world singularity a nation where no single dance, act or story or claim to countryhood is to be caught up in offensive and divisive space.
Why Canadian Music? But anyway that doesn’t seem to be the point. It’s interesting how it took the chance to juxtapose culture past and present and somehow capture the crux of the historicism that the bicentennial seemed to be going for.
Best response
In 1990, four young Singaporeans sent pictures to the British Broadcasting Corporation that speaks to what happens when a creative genius realises what his greatest gift is — by not only working and acting together, but all of these couples have found a way to connect to, and over the course of our lives, share their experiences and wonderful impressions.After the British Broadcasting Corporation finally gave us the option of just broadcasting a chat, a score, or a headline, the beautiful little cluster of individual pictures were re-listed as a free speech issue, a feature in the first issue of the Geographical Bulletin.This was a prime opportunity to portray how the industry has responded to extremist views and promote sensitivity in unique ways.
This was really difficult to choose amongst the 5. Granted there weren’t many samples but this one stood out to me as the most coherent. At least the themes seemed somewhat coherent, that was the aspiration of youth to share a piece of their perspective with the world.
Most ridiculous response
Watch my interactive animation below on How We Operated Our Island this December to get a glimpse of that story. Heart리리生 Sambit — Univers Paris 5 2014 UCC, Zida ng Mingin In 2014, I found myself standing in Lampedusa…and SA’d by hundreds of farmers and fishermen across the countryside. With the early stopping down and expanto…I remarked on my expertise as an epigraph and
No idea what happened here, maybe the AI just broke itself?
5. National Day Rally
What’s the biggest speech of the year? Every year the Prime Minister of Singapore delivers this speech as an opportunity to address salient issues at heart to every Singaporean. Let’s take last year’s transcript and see what the AI makes of it. The opening speech was very much a quick nod to the biggest event of the year, the Trump-Kim summit.
Good evening again. My fellow Singaporeans. We have had a busy year, both at home and internationally.
Two months ago, we hosted the first ever meeting between a US President and a DPRK Chairman, Mr Donald Trump and Mr Kim Jong Un. Singapore was just the host; we were not involved in the discussions. Still, we had a big responsibility to ensure that the Summit proceeded smoothly and safely.
Why did the US and the DPRK choose Singapore to host the Summit? I think they considered us a serious and reliable partner. We are friendly and straight with all parties. And both trusted us to have the infrastructure and capability to provide a safe and secure environment. It was a daunting task, but we rose to the challenge. We welcomed our guests with ‘Uniquely Singapore’ touches of warmth and hospitality. You may have seen the Sentosa cannons, barrels decorated with flowers and olive branches. SDC has good staff. We found and assigned Korean-speaking officers to the North Korean delegation. We packed goodie bags for 2,500 international journalists, with thoughtful items like bottled water and a portable fan to beat the heat. We kept the journalists well fed, with meals and snacks from local sponsors. We even had laksa flavoured cookies.
After the Summit, we held appreciation events for the officers who had worked so hard. I attended several of them. Tonight I want to thank the officers and your families once again, personally. You did not just answer the call of duty. You gave your best and more. Many of you worked 24/7. It was Ramadan, and Muslim officers were fasting. Some officers even had to abort leave plans, and cut short family vacations at the last minute. At the Home Team party, I chatted with a Gurkha Contingent officer, who had been on duty outside the St Regis Hotel. He told me how proud he had felt when a woman leaned over the crowd barrier to tell him: “Thank you for keeping Singapore safe.”
Full samples here
Most interesting
Thank you so much Penang! Later that evening, I walked to the 3 Western Seals Breakfast Ceremony. Thick peaks of ember were rising, and I was behind Pahang Palace when Rhee revealed his tall Chinese actor Iain Doyle. I enjoyed his athena, a full food table in beautiful Royal Place. I also described one police officer who stood guard in one metre adjacent to his body with his correct hand restrained so tightly. These officers pose on the table like statues, displaying their inner strength and power without detraction. When possible, I revealed to Sister Ang in China whose husband was the Chief Superintendent of Singapore Police’s General Information Department the stern image of officers barreling down to their subjects with pillows behind their heads as if investigating their crimes. One of those detectives, seated next to his grave was ALSO HOMELESS. Congregational Commission member, M.N. Masien Booth Gow, President Singapore’s National Communist Movement. As all revellers enjoyed our first MST, the Rho Al Hindi dinner concluded with laughter: I’ve always felt honored by that Meaty Export, .They are from Burma, we so miss that place… “Most Chef’d names.” It has been several years since the Association of Southeast Asian Nations published our Scorecard visited Burma, and thousands did share our heartfelt gratitude to the 11,000 food, fuel, necessary and inclusive humanitarian services available in the country.
At a certain point it looks as if all it was trying to do is show off how many people were met and how great that experience was.
Best response
On Day 15 inclusion of Singapore at the Summit made it clear that we do today how sad it is the responsibility of all Singaporeis to hold new friendships during Ramadan so they can create even more sets of bonds which strengthen world-historic organisations, heal wounds and uphold human dignity, diplomacy and unity. Everyone across the world are recognizing It is important for us — South Asians, African, Cambodian, Tibetan.Singaporeis already hold the global fame for surviving and in effect continue to do so. And remember, we are also a problem. No linguistic restriction; no seat-in-nation and even less welcoming treatment is discussed. Canadian-Chinese unity crisis appeared to happen at the peak. President Bali’s death came as a result of a hostile campaign to stone Manchuria Dalai Lama. Amicable diplomatic support and peaceful meetings with Tibetans aside, as real problems have arisen affecting our interests and national interests, I am walking a tightrope, trying to reach something that does not exist. Together we achieve peace and there are tangible costs of our decisions to micromanage our society. Visa holderships close to 70,000, and private secretary collections of almost ¥122 million. While heartbreaking for all, as diplomacy could only progress so far with ideas, maths and science, it could also prove to have lots of value beyond the social support. However, future success will bring us more challenges and symbolism, further isolating the values in our society. Is there any Opportunity? I tell you, it provided more for all Singaporeans — hard-working and graduates — we now learn from Astride.
This is taken from a snippet of sample 3. It seemed like it was interpreting that Singapore was hosting the summit to get a seat at the table of international negotiations. I think the ability to understand context is still much to be desired.
Most ridiculous response
I also met the Deputy Chief of Army Staff, Senior Deputy Chief of Army Staff, Special Army Staff, Special Brigade Staff. These are all good officers. For those of you coming after Singapore should join our rowing unit in our inaugural training. The Royal Navy invites you to attend my reexamination of the historic choke hold called the Wing Pad. SAAMI AMPLE BED312–558 Turcy steering wheel undercarriage; C drone-armed Kudu tyres; 3 Mk Manila 61A; gimbal mounted on hull; 3 plate SOD-2 Potter basket boat with Piranha eight- and 10-gun Sqdn rockets aboard 13 min boat. 2 Exp reprinted with mod. 2014 Cross-posted as Singapore Diplomat<|endoftext|>To those of us who haven’t experienced it it can get confusing at times, but this technique works for you because looking at the diagram below, there is no climbing, air or ground spirit. The elements feel concentrated, you grasp them open for air ascent, and then, your foot moves into a barrier, which is withstood by your arm, muscle rosin and nausea and vomiting.
To get simple:
Balance your body one foot to repeat 31 great steps on eight legs
Repeat 33 great steps twice on eight thighs
Talents and Lessons
And she is capable of a lot of sumo-work, such as alternating each step permanently, easing the way yourself, tracking your time with warriors, literally pulling you back to your feet and throwing you all sorts of sphinxes (such as mini-Shiba, feet rolled out as if to place hamstring in front of your left index finger).
Eroscillator/Magic Stretching:
When you learned this with Mundeke’s Kung Fu toedles, I enjoyed it for not just trying and crunching but not only hurting but losing charge of my muscles. After having it take me the stand experimentally, I learned to use quads as if they were button joints to bring me back to agreement with Mike Probably’s best to have your fifth finger in sphincter has copiously danced down another of his pins and then my feet, and now with them, giving you dimensionally no cues.
To use, take your finger on your toes and bend your no-holds-barred pillar, spinning a second time around. Finger-turn with your thumb back adjusts the angle of your back shoulders so your extra hips at the back of your neck are
It went completely mad! It’s so hilarious because it started out quite well with the titles and the meetings and well it went completely overboard with the kung fu moves.
Conclusion
Compared to the samples put up on the blog, I think it’s quite clear they released an extremely nerfed version. Even so, this looks like quite a feat.
- It seems to recognise broad themes but tends to get distracted quite easily a short way into the sample.
- When it gets distracted, it gets REALLY distracted.
- It might misinterpret an article’s tone and purpose and end up writing something else completely.
- Got to get that puctuation right.
- I think as a person, it might be quite easy to over-read into what the sample is trying to say to give it benefit of the doubt. The results are reminscent of the Chinese Room argument as it doesn’t seem to know what exactly it is doing.
