The name “Jack O’ Lantern” was originally one of the numerous names given to ignis fatuus (Medieval Latin for “foolish fire”), another of which is “Will O’ the Wisps”, basically the odd light that can occasionally be seen over marshes, swamps, and the like. “Jack O’ Lantern” first popped up being used this way around the mid-17th century in East Anglia, UK and spread from there through parts of England, Ireland, and Scotland.
This name likely originally derived from the practice of calling men generically “Dick, Jack, Tom, etc.” In particular, men who were lower class, were often called generically “Jack” beginning around the 14th century in England. (This practice popped up again in the late 19th century in the United States where “Jack” was used for the name for a man where his real name is unknown). Thus, when you see someone carrying a lantern in a distance at night that you see is a man, but you can’t make out who exactly it is, he is literally “man with a lantern”, a.k.a. “Jack of the Lantern” or “Jack O’ Lantern”. This was also commonly used for a nickname for night watchmen and the like around the same time “Jack O’ Lantern” first popped up, referring to ignis fatuus.
As to how this name made the jump to referring to carved pumpkins with lights inside, it has its origins in the Celtic practice of hollowing out and carving faces into turnips and other vegetables during Samuin (a festival where many of the traditions of Halloween come from). After carving the vegetables, they’d place candles inside and put them in windows or carry the make-shift lanterns with them as they walked to ward off evil spirits.
The tradition in various forms endured, including in Britain where pranksters would make these types of carved lanterns to scare people on the road or children would carry them around during Hallowmas while begging for soul cakes.
From here things get a little murkier in terms of tracing how exactly “Jack O’ Lantern” got applied to carved pumpkins. It is entirely possible that the above “man with a lantern” nickname of “Jack O’ Lantern” may have simply directly extended as a nickname for these carved turnip or vegetable lanterns without any intermediary. The first known instance of this application was in 1837 in Nathaniel Hawthorne’s The Great Carbuncle:
Hide it [the great carbuncle] under thy cloak, say’st thou? Why, it will gleam through the holes, and make thee look like a jack-o’-lantern!
On the other hand, given that it appears it was Irish immigrants to America who brought over the practice of carving and lighting vegetables and it appears it was they that adapted this practice to carving pumpkins for this purpose, it seems likely enough that the nickname did not make a direct jump from “man with a lantern” to “carved vegetable lantern”, but rather came via an Irish legend, Stingy Jack. (This legend sprung up around a century after the first known instance of “Jack O’ Lantern” referring to ignis fatuus.)
There are numerous versions of The Legend of Stingy Jack, but the general story is there was a sinful drunkard named Jack who encountered the Devil and convinced the Devil to come have a drink with him before the Devil would take him down to Hell. The Devil agreed and after Jack had drunk his fill, as he had no money, he asked the Devil to cover the bill. As the Devil had no use for money, he had none on him, so Jack suggested he transform himself into a silver coin so Jack could pay the bill. (Why Jack or the Devil would care about paying at this point in the story makes little sense, among other oddities, but shhhh! You’re interrupting the story.)
Once the Devil had transformed himself into the coin, rather than pay the bill, Jack quickly stuck the coin into a small money pouch which had a crucifix on it, trapping the Devil.
In order to get Jack to let him out, the Devil made a deal with Jack that he’d give him 10 more years of life. The bargain was struck and after 10 years, the Devil returned for Jack and Jack made the request that the Devil fetch him an apple from up a tree. The moronic Devil, who apparently lives to please, climbed the tree and fetched an apple for Jack. While he was up the tree, Jack carved a crucifix into the bark at the base of the tree, thereby trapping the Devil… who could apparently turn into a coin but couldn’t turn into a bird. Or, you know… couldn’t just hop down past the crucifix instead of climbing down when he reached that point…
A second deal was struck, this time with Jack demanding that the Devil never take his soul to Hell. The Devil agreed and, being known for honesty and integrity, kept his word once Jack let him go.
Of course, Jack eventually died (maybe he should have asked for immortality and eternal youth?) and upon arriving at Heaven’s gates, whoops! His sinful life made it so he couldn’t get in. He then went and visited the entrance to Hell to try to get in there (for some odd reason), but the Devil wouldn’t let him in owing to his previous promise. Being once again apparently kind and generous, the Devil did give Jack an ever burning coal from Hell to use to light his way / warn other people of Jack’s presence. Jack then placed it in a carved turnip (which apparently doomed souls have easy access to) and proceeded to wander the Earth for all eternity.
So whether it was a direct jump from the “man with a lantern” British slang to being associated with people holding vegetable lanterns and then the vegetable lantern itself or, probably more likely given how vegetable carving became popular in America, via the Irish Stingy Jack legend, the ultimate origin of why we call carved pumpkins “Jack O’ Lanterns” seems to be a melding of the ignis fatuus nickname, “Jack O’ Lantern”, with the Samuin practice of using hollowed out vegetables with candles inside to ward off evil spirits.