Outline

• Introduction and background
• My first prompt
• ChatGPT-3.5 result
• Time investment and return
• Evaluation

Introduction and background

In this multi-part series of articles, I will document my efforts in writing a high school Philosophy paper using ChatGPT. The paper shall answer in writing a question I was asked at an impromptu oral assessment in Philosophy class in my 3rd year of high school.

Let me take a step back to give you some context. I was enrolled in Classical High School in Rome, Italy. This type of school has a strong 5-year emphasis on History and Philosophy, as well as Latin, Greek, and Italian literature and grammar, with a bit of Science and Math. Students are typically evaluated on weekly written assignments, a minimum of 3 scheduled written tests each semester, and a minimum of 2 oral assessments per semester. During the latter ones, the teacher would (usually) pick two names at random from the attendance list once a week and gauge their knowledge on both the weekly lesson, in detail, but also on anything covered in the year up to that point; this could last between 30-60 minutes, and would happen in front of an audience of (often roaring) classmates. Seriously: brutal!

Only occasionally were students allowed to give a presentation on a prepared topic or volunteer for a full oral evaluation. This was the case for me during a block on Plato. And so it was that the teacher, the famously feared Professoressa Carbone (who taught both History and Philosophy), probed into my soul with her terrifying eyes and asked with a sinister smile, “Tell me, Niccoli, what would Plato have thought about the Hundred Years’ War?“

My first prompt

When I heard about OpenAI ChatGPT being available, it seemed very fit to try this question in my first investigation. So I went ahead and asked the question “What would Plato have thought about the Hundred Years’ War?“, with only the minor modification to add citations and break down the response in steps.

GPT-3.5 result

Below is the result from the default GPT-3.5 model:

Time investment and return

• Time investment: the whole investigatyion took about 30 minutes, including a very light read on prompts from a reputalbe source, and reading the answer (becaue the citations really did not come as full references, there was really nothing else to do other than that).
• Return: was this worth 30 minutes of my time? The answer is, as often, it depends! On the one hand, this is a really poor answer in my view (see next section for a more detailed evaluation). So, as an essay (l imagine myself as a student looking for a quick free lunch), this would not do it; not even close. For me… but then again, even at 13 in my first year of high school I would’ve known this was not going to cut it… or in Junior high, for what matters. On the other hand, I was deliberate in providing a very simple, not engineered or iteratively refined prompt. As such, the response will come useful as a benchamrk against wich I will compare other efforts (for example using a better model, GPT4, and improving the prompt).

Evaluation

Is the answer grammatically correct? yes; is it written in a uniform style, appropriate for a school essay? yes; Does it contain any hallucination? I would say no, everything sounds reasonable. However I must admit that without reference to specific passages to Plato’s work, it is hard to say for sure. From what I remember of Plato’s Philosophy it’s reasonable, and that’s all.

On the other hand, had I been handed in this as a teacher, I would not be impressed. IF this were the output of an in-class exam, I probably would have marked it as a D- (a D for the very mediocre text and a minus for the ridiculousness of the citations). Had it instead been handed for an at home written assignement (in which cse the student would have had access to their notes and books) this would definitely be certainly worth an F (Fail). Now I sound like Professoressa Carbone.

You may reasonably wonder at the end of this article: how did I fare in that evaluation in 1986? Fair question. And I am not ashamed to confess it was a disaster. At 15 I was still very immature, intelligent but unengaged in school work, and also unable to do a real analysis. By that I mean interpret information rather than just accumulate facts and connect them. I honestly do not recall the mark I got (we had different levels of Failure that went progressively deeper, not too dissimilarly from the Circles in Dante’s Inferno) but I rememberr that an F definitely it was.

P.S. The model did a perfectly fine job in proofreading my first draft for this post. THat was definitely worth a lot, given I type very slowly, and I make lots of typos.