Can I change the colors (background, axes, etc.) of the plots in WEKA?

Sure, this information is stored in the Visualize.props properties file:
  • weka.gui.visualize.Plot2D.axisColour defines the color of the axes
  • weka.gui.visualize.Plot2D.backgroundColour sets the background color
For more information see the articles about properties file (especially the section Precedence will tell you where to place the .props file.) and Visualize.props itself.