You can think of .adp as a web page with embedded Tcl code, and .tcl as the reverse (Tcl script with embedded HTML/text output).
So you can use either one, but if you have more HTML than Tcl you'll find it more convenient to use .adp, and vice versa.
If you're writing .adp, you have to enclose the Tcl in special tags. If you're writing .tcl you have to explicitly write the HTML/text output to the client connection (eg. using ns_write. You have to remember to quote the double quotes within .tcl output.
The aolserver web site has more details.
Good luck with the problem sets.