Here is a quote from an article on the impeachment of Trump:
Prosecutors can only bring charges if they have evidence that the criminal is guilty. But even then, not every criminal act gets prosecuted — or needs to be. Prosecutors must weigh the public consequences of a prosecution. They must ask themselves whether there are mitigating factors that weigh against bringing charges even against someone they believe to be guilty. Prosecutorial discretion is a crucial component of the job of being a prosecutor.
Similarly, congressional leadership has the responsibility of exercising what we might call impeachment discretion in the case of a president. Not every constitutionally impeachable act can or should be impeached. Judgment is required.https://www.bloomberg.com/opinion/articles/2019-12-10/trump-impeachment-articles-attempt-to-keep-it-simple
Prosecutorial discretion is a case of operational autonomy within the legal system. Subsystems within the legal system have been granted a degree of operational autonomy. Without this autonomy, the legal system would work like a machine, with no flexibility. But some people argue that this principle permits or invites arbitrary justice. For instance, Angela J. Davis and Joy Ann McDougall write,
Prosecutors are the most powerful officials in the criminal justice system. Their routine, everyday decisions control the direction and outcome of criminal cases and have greater impact and more serious consequences than those of any other criminal justice official. The most remarkable feature of these important, sometimes life-and-death decisions is that they are totally discretionary and virtually unreviewable. Prosecutors make the most important of these discretionary decisions behind closed doors and answer only to other prosecutors. Even elected prosecutors, who presumably answer to the electorate, escape accountability, in part because their most important responsibilities— particularly the charging and plea bargaining decisions—are shielded from public viewMcDougall, J. A., & Davis, A. J. (2007). Arbitrary justice : The power of the American Prosecutor. Retrieved from https://ebookcentral.proquest.com
But they go on to state that this principle is a “necessary evil”:
Prosecutors certainly are not the only criminal justice officials who make important, discretionary decisions. Discretion is a hallmark of the criminal justice system, and officials at almost every stage of the process exercise discretion in the performance of their duties and responsibilities. In fact, without such discretion, there would be many more unjust decisions at every stage of the criminal process. A system without discretion, in which police, judges, and prosecutors were not permitted to take into account the individual facts, circumstances, and characteristics of each case, would undoubtedly produce unjust results.
Police officers, for example, who are most often at the front line of the criminal process, routinely exercise discretion when making decisions about whether to stop, search, or arrest a suspect. Although they are permitted to arrest an individual upon a showing of probable cause to believe he or she has committed a crime, they are not required to do so, and frequently do not. A police officer may observe two individuals involved in a fistfight. Such an observation provides probable cause to arrest the individuals. Yet the officer has the discretion to break up the fight, resolve the conflict between the individuals, and send them on their way without making an arrest. Such an exercise of discretion may well be in the interest of justice for all involved and would save the valuable resources of the court system for other, more serious offenses.p. 6
Discretionary power can always be abused, and when abuse is detected, other systems might intervene. Judges, like presidents, can be impeached, for example. But this fact doesn’t negate operational autonomy.