Some debates are so timeless, spanning as they do from ancient India and ancient Greece to the European Enlightenment and the modern internet debate forums, that it becomes difficult to avoid noticing patterns in the arguments.
I have noticed five distinct categories of arguments used by atheists and theists, some of them much better than others. They range from truly awful to astoundingly mediocre to quite sensible. I have organized them in tiers according to how useful or effective I think they are as a category. My purpose here is not to evaluate individual arguments, but to elucidate the qualities of various types of arguments.
Tier 5 - Snark and Ad Hominems
This tier is the worst of the worst as far as the potency of the arguments is concerned, mostly because they aren't even arguments at all. It is however the most entertaining and "feel good" part for those of us who like to openly indulge in a lot of bombastic confirmation bias. At this level of argumentation, witticisms are bandied about, fun is poked at various parties, and insinuations about the intelligence of one's opponents are ubiquitous.
If you find yourself entertained by these sorts of exchanges, you needn't feel that there is anything wrong with that. If you find yourself persuaded to one side or the other by them, then it's time for you to learn how to think critically. If you feel the need to use this sort of "argument", my recommendation is to sprinkle them lightly into your discourse. A little bit of it adds flavor, but much more and it ruins the dish.
Tier 4 - Obviously Fallacious Arguments
This tier is one of the most popular places for folks to park their intellectual abilities. The arguments are easy to understand for those who make them and easy to refute for those who disagree with them. Everybody gets to feel very competent and impressed with themselves at this level so long as they don't actually understand argumentation very well or just conveniently ignore it when it suits their purpose.
An example of this type of argument would be the extremely common case of question begging. For theists, it might be the argument that we can conclude that God exists because the Bible says so. What they leave out is that they believe that the Bible is authoritative because God inspired it. Whoops! For atheists, it might be the argument that science has never discovered evidence of God, which means that we can conclude that God doesn't exist. What they leave out is that methodological naturalism and principle of parsimony make it impossible for science to ever conclude that God exists no matter what the evidence might be. Whoops! These arguments just flat-out assume what they're trying to prove, a time-honored custom in human cognition.
In both cases, these arguments are for people expressing their confirmation bias boldly without realizing it. At least they're making an argument, but we should really encourage them to do better.
Tier 3 - Classical Logical Arguments
This tier is where a lot of folks who have a basic grasp of logical argumentation and fallacies tend to spend most of their time. Premises and conclusions are set forth, syllogisms are employed, and terms may even be defined. Definitions are disputed, reductios are performed, and the laws of classical logic invoked. This is what the folks at Tier 4 probably think they're doing, but haven't quite managed to accomplish.
Some examples of this type of argument are the Problem of Evil and the Cosmological Argument. Correctly stated, these arguments are logical in form. This does not mean that their conclusions can determine what exists or that these arguments are impregnable, and particularly when we critique an argument we have to be careful of making an argument from fallacy.
These arguments are actually arguments and they are logical, but they are not demonstrative of anything but the ability of the person formulating the argument to work within the structures of classical logic, structures which (as Krauss has pointed out) we cannot use to model our world and consequently predict what sort of things exist because quantum mechanics violates the fundamental axioms of classical logic.
If you're looking for an interesting intellectual exercise or an opportunity to practice critical thinking skills and maybe try out a new argument, then this a great place to be. If you're looking to prove something and you take yourself really seriously, then I recommend going deeper into the rabbit hole and learning how little you know.
Tier 2 - Epistemology & Ethics of Belief
This tier is where folks start exploring the limits of what we can know and the propriety of our belief formation process. Here we can take positions on how we ought to decide what to believe, wonder whether or not we should believe what we cannot know in a rational sense, and decide whether or not we can know that God exists or that God does not exist.
The debate at this level does not hinge so much on argument as on intuitions and sensibilities. At this tier, the players explore the axioms rather than the arguments, the claims necessary for making the argument, and the values which shape our worldviews. There are a number of common questions we seek to address at this level. What constitutes a good reason for believing a claim? Do we need to know the claim in the sense of meeting the requirements of the Tripartite definition of knowledge to be justified in believing it? Do we know things empirically, rationally, both, or neither?
Here is where you will notice claims like the following. "It is wrong to believe anything upon insufficient evidence." Or perhaps, "Extraordinary claims require extraordinary evidence." These are platitudes rather than arguments, but they are not without weight. You'll find a lot of agnostics hanging around this place, stuck in endless questioning and understandable uncertainty. Some of them are unwilling to take a stand on the theism question and others are willing to jump into the fray on one side or another.
Most people never venture beyond this tier, and even fewer do so comfortably. The benefit of this tier is that, assuming you understand what happens here, it helps you to understand how limited our intellects are and mitigates our natural hubris. If not, then you'll just be a slightly more educated person who still doesn't understand their cognitive limitations, which is probably a lot of the folks who visit here.
Tier 1 - Problem of the Criterion
This tier is a very sparsely populated one, and folks tend to pass through more than anything else. This is where we have to formulate a standard of evidence which can allow us to adjudicate claims about theism without appealing to standards which assume our conclusions and which we can apply usefully to our beliefs. Neither theists or atheists manage to do this very often. Even the most intelligent among both groups tend to find it very difficult to apply genuinely neutral principles.
The question we seek to answer here might be posed this way... "What impartial principle can help us decide the matter?" The reality is that most of us don't use impartial principles to form our beliefs because that's not how our minds naturally work, which makes this a tough question to answer in practice.
If you're willing to get to the rock bottom of your worldview and spend some quality time within your own mind cleaning house and cogitating robustly, then this is the place for you. If not, then just enjoy your confirmation bias and whatever intellectual pretensions you have managed to collect. If nothing else, you can do the popular thing and feel intellectually superior to everyone else without good cause.