[Darryl Ramm]

Your experience is likely about normal.. you have to work out how to do this. And you need to start simple, follow some guidance/tutorials and spend a *lot* of time playing around.

Lots more useful info needed: how you are listening back to what you have recorded? What exact monitors and/or speakers or power amps? and/or headphones? What type of tone are you going for... do you know what that artist used? What amp or the models in the Katana or Eleven that you have tried? What levels in Pro Tools in dBFS did you record at? (make sure you are in pre-fader meter mode). You should be aiming for peaking at something like -18dFS. Something as a beginner especially it will help you avoid any input clipping, and with the low-end electronics in the solo will not sound good when pushed hard... and be sure you have the Line/DI switch to line position. It sounds like the line input gain can be adjusted with the line level input (which makes me think you might not have it in line input mode), but check see what the input is in Pro Tools in dbFS (remember you have to be in pre-fader metering mode).

Is your room set-up, roughly calibrated to a play-back level? (seriously, yes even a home studio, ~75 db SPL-C is worth trying for if your monitors are capable of enough SPL, and that's the huge question here. Trying to calibrate stuff or hearing recommended mix SPLs levels give you an idea of the environments that professionals mix in.).

There is a huge difference between the sound and feel of an amp in the room and an a wet guitar signal recorded and played back though some monitor/amp/speaker chain. And that starts with a large perceived difference due to different SPL (sound level). Google "Fletcher–Munson". And then (as here) an open-back speaker cabinet spraying sound all around a room, reflecting off walls, etc. is a whole different thing that listening back on studio monitors.

You may have got too much going on here, especially for a beginner. You are using a combo modeling/power amp and feeding into a DAW, and are you then playing with adding Eleven amps sim plugin to that signal? Or are you only using Eleven plugins within the guitar going direct into the DI input? And you are trying to do all that with maybe inexperience with basic recording and no mixing/post record processing being done. Play around but with that setup and work out what you like the recorded sound of most.

You will likely want to do some signal processing early on, not much but some. You may need to adjust the input track fader to adjust its level into any plugins on that track and tweak the plugin settings themselves. You may want EQ plugins to sweeten things, maybe some reverb on that track. Then on an output bus a gentle compressor, or compressor followed by reverb etc. But the point is you likely will want to sweeten the sound a bit while tracking (if you monitor though Pro Tools) or when listening back.

Recoded guitar tracks are usually *heavily* processed, EQ'ed, compressed, may have reverb added, often multi-tracked, or reamped or reamped into multiple tracks. etc. And then the whole mix is mastered with EQ and compression applied. Try to find some of the videos on you tube where folks involved dissemble popular hits and you likely will notice how thin the guitar tracks sound stand alone.

I would pick one core thing and play with that, explore it until you understand it fairly well (~you play with stuff over several days for hours per day) and then move on to the next. So pick if you are going to start using your external amp or if you want to plug the guitar directly into the Solo DI input and use the Eleven plugin. Try to find a tutorial on YouTube for somebody doing something similar to what you want to do with that amp or Eleven plugin and follow along with that tutorial to start with.

You will likely get more help here if you go with the Eleven Plugin, lots of us use that, and/or have Eleven Racks. But pick what you want to do.

The Katana might or might not just sound bad out the line output, I've never heard one. Run the line-out into a loud enough PA system (to get high enough SPLs) and listen to it through that. Since it's a digital modelling amp I would hope they try to do a pretty good job, including I hope they have a speaker-cabinet resonance/EQ and maybe mic simulator processing applied to that output. Hopefully they explain that in the documentation, but being Boss I'd not be surprised if the documentation is a pretty bad Japanese translated mess, they don't even seem to want to admit it's a modeling amp from what I see. If the line output is missing cab/mic simulation you may want to make up for that in Pro Tools.