The thorn looks like this: Þ or þ or .
When printing presses were first set up in England in the 1470s, the type and the typesetters all came from Continental Europe, where the thorn letter was not in use. The letter y was used instead because in the handwriting of the day the thorn was very similar to y (if you're wondering why they didn't use p, it's probably because p was used for another Germanic rune; wyn**).
So all those quaint shop names beginning "Ye Olde" are based on a confusion: people never said "ye" to mean "the." However, if you'd rather be cute than historically accurate, go ahead. Very few people will know any better.
*The other ``ye,'' the plural of "thou," has a separate, less exotic etymology. The word ``you'' is not a misreading of ``thou.''
** Wyn =