For your visit half a day should be enough to get acquainted with the city (but palace visit will take at least two hours).
Distances are very short in the town center.
Worthy attractions: Palais des Papes; Pont St-Bénézet with tiny Chapelle St-Nicolas.
Left for another visit: Petit Palais; Cathédrale de Notre-Dame-des-Doms [did not tour inside]; Musée Calvet; Musée Lapidaire.
Last visit: April 2002.

As is our custom on day-trips, we spent most of the time wandering around city streets on our only visit to Avignon. It is a very pleasant town, with many splendid houses found within its historic core. It has several interesting churches and museums as well, but we limited ourselves only to the couple of most famous ones.
Palais des Papes ♥♥♥, where the Curia resided for most of the 14th century, is massive and even threatening, and unfortunately practically empty of the furnishings and works of art. But a number of its rooms are utterly impressive and a couple of vast grand halls are nothing short of breathtaking. There is an excellent audio-guide that talks about the construction of the palace and of the history of the Papal Court.
The remains of the renowned Pont St-Bénézet ♥ can be toured with another audio-guide, which gives insight into medieval history of Avignon. The tiny chapel on the bridge is an interesting curiosity. The Papal Palace appears very imposing when viewed from the bridge.
Places to Eat
We had a conceivably good dinner at Le Forum, but in the absence of specific notes, I cannot make a firm recommendation. It is situated on the central Place de l’Horloge, which is surrounded by various eateries, and we picked it fairly arbitrary, so I suppose you cannot go horribly wrong in that locale.
In our wanderings around town, we also stopped for a break at a tea shop somewhere on a quiet street, as well as had a light lunch in a café of the palace square (a bit overpriced due to location, of course). We did not record specific names in our diary then, but recall the experiences as very agreeable.
Pont du Gard
You will undoubtedly be awed by the first sight of this 2000-year-old aqueduct ♥♥♥, one of the best proofs of the greatness of the Roman Empire. The Romans themselves considered it the best testimony to the greatness of the empire, and it continues to majestically span the river Gardon after all these years.
At time of our visit, access to the top level of the aqueduct was closed, so we cannot attest to the splendor of the view from the top.
While technically in Languedoc, the aqueduct is located less than 30 kilometers away from Avignon. You don’t have to plan on more than an hour for the actual visit to Pont du Gard, unless you want to have a picnic in its sight (which is what a large segment of visitors do).