¶

Python vs C++ (even C++20) and C# in context of properties, scientits,best C# datatypes, etc.¶

I compare python to C++ and C#, particularly for scientists, with a very unbalanced emphasis on the issue of encapsulation, allowing a low initial coding investment (which scientists love) while enabling easy code re-use and maintenance later (which scientists need).

For non-programmers who need to program, I think some organization in code is better than none. One tool is grouping variables with functions into classes or objects, where their scope of effect is more limited and clear. While encapsulation is then ideal, direct access could be more familiar.

Properties are supposed to resolve that, and advertised a lot for that for Python (but should they be?), allowing simple code early, that can be shared and benefited from, while allowing it to be encapsulated later with less disruption to anyone using it. And for this reason it's an important feature to me in comparing languages for collaborative non-programmers.

Removing concerns over future access control seems to be really driving a lot of modern programming exactly, in part, because it makes it so much easier to benefit from reusable code, easing library development and sharing. This shows up in the trends toward immutable objects and functional techniques which avoid the issue entirely.

So I will start with the more technical discussion on properties and end with some thoughts I've collected comparing C++, python and C# in general.

Class attributes and maybe one reason C++ turns off non-programmers¶

We eventually learn we should use getter and setter functions just in case some day we need to add checks or corrections to a variable or often to other related variables, whenever one is changed. And if you don't have them from the start, adding them later will often change how your code has to be used, everywhere, often resulting in diverging versions of the code.

In C++ this takes some typing, unless you autogenerate them with any nice modern IDE (not vim). Here's a simple enough example. Start with a quick "script".

In [20]:

%use xcpp17
#include <iostream>

using namespace std;

class Calibration {
  private:
    int _offset;               // Private attribute

  public:
    void setOffset(int o) {     // Setter
      _offset = o;
    }

    int getOffset() {           // Getter
      return _offset;
    }
};

//int main() {    // no main in jupyter
  Calibration myObj;
  myObj.setOffset(50);
  cout << myObj.getOffset();
  return 0;
//} 


// Ignore error below here, it's a kernel setup bug:

Out[20]:

"All this noise just to set and print ONE variable!!!! ARGH!!!" I sympathize with this sentiment.

This turns off non-programmers from even collaborating on this code, let alone writing it. (In reality, modern IDE's generate the setter and getter code automatically and it just takes getting used to but....)

Python to the rescue!!!¶

In python we can just make class attributes, and assign to them and read to them like variables, and IF we later need getters and setters, we can convert the attribute to a property later without changing usage syntax. Or this is how it's advertised anyway. It isn't really true. Let's see.

A simple class:

In [2]:

%use python3
class myclass:
    def __init__(self):
        self.my_list = [0,0,0]
        
    def print(self):
        print(self.my_list)    
#Main:    
obj=myclass()
obj.my_list=[-1,2,3]   # set the whole list
obj.my_list[2]=-1      # or set an element.
obj.print()  # 1,2,-1obj.print()  # 0,2,-1

[-1, 2, -1]

Life is good. No setters/getters, just simple classes, simple access, let's us write code fast.

... Until one day you want to always make sure that -1 gets "fixed" to be 0.

This is SUPPOSED to be a GREAT thing about python. In C++ you should have used getters and setters from the start to do obj.set_mylist([1,2,3]) or obj.set_element(2,-1) just in case some day you need it. You didn't?! You're out of luck. Adding it now will break all of your code. Everywhere you, or anyone sharing your code, said obj.my_list[2]=5 now needs to be changed to obj.set_element(2,5) or similar.

(This is a barrier to sharing code openly, so efforts diverge and duplicate instead of building. But you can't program well if you're always worrying about it, which is why properties exist.)

In python we can just add it in later without needing to change how the class is used everywhere... But can we really?

This is the same single class, now the with the single public member upgrded to a property with a setter to "fix" things. You can skim the next code, but the pont i this works if/when we need it to, without changing usage everywhere:

In [22]:

%use python3
'''         ***********Python setter added to change BAD -1 into GOOD    0 ************'''
class myclass:
    def __init__(self):
        print("Init")
        self._my_list = [0,0,0]
        self._setter_num_calls = 0  # just for debugging
                        
    def _get_my_list(self):
        return self._my_list

    def _set_my_list(self,list):
        self._setter_num_calls += 1 
        print("Calling setter ",self._setter_num_calls," times")  # debugging output
        for i,val in enumerate(list):   # our setter can catch bad values!!!!!!!!
            if val == -1:
                list[i] = 0
        self._my_list = list
        
    #It's more common to use @property notation instead, but it's up to style:
    #     (they do exactly the same thing). 
    my_list = property(_get_my_list,_set_my_list)  # Now a property 

    def print(self):
        print(self._my_list)    

#Main:
'''***************No changes needed to user code!!!***************   ''' 
obj=myclass()
obj.my_list=[-1,2,3]   # Assigns a new list.  Calls setter 1 time
obj.print()  # [0,2,3]   # -1 caught by setter and converted to 0.

Init
Calling setter  1  times
[0, 2, 3]

The bad -1 value got intercepted. This is great until this happens:

In [4]:

%use python3
obj2=myclass()
obj2.my_list=[-1,2,3] 
obj2.my_list[2]=-1         # Assigns a new value/object to SAME list.   *****bypasses setter!!!!!!!!
obj2.print()  # 0,2,-1

Init
Calling setter  1  times
[0, 2, -1]

:( That bad -1 still got in!

Assignment to the list variable is intercepted, but not assignment to the elements in it (or addition or deletion)!!

The property is almost useless.

There are two parts to the problem here:

We let users do whatever they want with data before adding the property, assuming we could worry later, and
now we cannot intercept that access later when we care.

This is WORSE than C++, where:

yes, we need the getter from the start or to break code later. It has no properties.
But at least its getters do normally prevent this, because it returns by copy by default, or (recently) constant views by request.

In C++ modifying the returned copy doesn't alter the object. If we need to prevent copying large objects, C++20 (like C#) can now return const views called "spans"(in C++). These are compiler-enforced lists of const references with no runtime penalty. In python, objects are either immutable, everywhere (inside and outside the class) or modifiable everywhere. There are no const views.

In python you can return a deep copy from the getter, but it's not default or cheap. It's not a free compiler-enforced views. Even with C++ spans though, a sub-attribute could have a non-const reference.

A class returning a container of objects doesn't have to deal with that though. The contained-object can themselves have setters on their attributes. That's fine unless we need context-aware setters. In any language, that's at best tough to deal with if modification was allowed from the start, but C# has some tools for that. Of course you can make the object immutable, which python can do, but that's not the same as forcing all changes through class methods now or later.

The idea that properties shield us from future concerns is false, but python's are particularly lacking, as we can see comparing to C#. And C# makes it a little easier to implement them or other gaurds from the start.

Let's try in C#

The fundamental problems above don't change. It's also true that C# essentially passes most objects by reference (not copy, but structs and records are useful exceptions), and doesn't in general have const references like C++, but for lists, it basically does have const ref.

It also has "ref readonly," but that still only prevents assignment to the list, not the elements, and a setter can already intercept that anyway, so this doesn't add much.

But C# does have some powerful redeeming features that help:

Its property syntax can be shorter, making it less overhead. In fact you're not meant to convert fields/attributes into properties in C# (not 100% compatible) but properties with default getters and/or setters are just easy to write from the begginging.
It CAN return readonly lists, without copying, where at least the list elements are not re-assignable, although attributes of objects elements still may be (depending on the object). This is already enough for lists of values/strings/etc, just not mutable objects. Python already cannot handle that case.
Like Python, you can also start with or add setters for re-assigning elements.
It does have good library tools for notifying changes to collections. Python could have this, and you can write it, but....

Let's look at combining the first two points, but start simple. If we only need to initialize an immutable object for now we can start simple and still add functionality later.

In [21]:

%use .net-csharp
using System.Collections.Generic;

public class MyClass{
    // Public readonly list of readonly elements.
    public IReadOnlyList<int> MyList { get; init;} = [];  // no setter, and list can't be modified.

    //But still can use initializer, and/or optional constructor param:
    public MyClass(List<int> list = null){  
        this.MyList=list;     
    }

    public void Print(){
       Console.WriteLine(String.Join(" ",this.MyList));
    }
}

var obj = new MyClass{MyList=new List<int>{-1,2,3}};  // use initializer
obj.Print();  // -1,2,3
var obj2 = new MyClass(new List<int>{4,5,6});  // or constructor
obj2.Print();  // 4,5,6

-1 2 3
4 5 6

Python competes here. It could just use a raw tuple (immutable). But it can't prevent users setting it to a new tuple without a bit wordier property synntax than this to define the backing field and the getter. C# is more concise. If we (now or later) want the class to be able to modify the values in the list, we can add a mutable backing field, without breaking anything, so we return a readonly view (not copy) but can modify it internally:

In [27]:

%use .net-csharp
using System.Collections.Generic;

public class MyClass{
    //Public readonly list of readonly elements:
    private List<int> _myList;
    public IReadOnlyList<int> MyList  => _myList;  

    public MyClass(List<int> list){
        _myList=list;      
    }
    //.. some methods that change _myList...

    public void Print(){
       _myList[0]=10;     // now we can change the list in the Class!
       Console.WriteLine(String.Join(" ",this.MyList));
    }
}

var obj = new MyClass(new List<int>{-1,2,3});
obj.Print()  // 10,2,3

10 2 3

At this point, python cannot compete. If you started with a tuple to prevent external changes, you can't make internal changes without copying every element of the result to a new tuple, or exposing a mutable list that can be directly modified externally. C# wins here hands down. To be fair, let's see the C++20 version of this that also does well, and in a similar way. My Jupyter won't run C++20, but here's the code:

In [25]:

%use xcpp20
#include <iostream>
#include <vector>
#include <span>

class MyClass {
private:
    // ************ Private mutable vector **************
    std::vector<int> _myList;

public:
    //************Public read-only view**********
    std::span<const int> myList() const { return std::span<const int>(_myList); }

    MyClass(const std::vector<int>& list) : _myList(list) {}

    void Print() {
        _myList[0] = 10;   /********  Internal modification allowed  *******/
        for (const auto& val : myList()) {
            std::cout << val << " ";
        }
        std::cout << std::endl;
    }
};

int main() {
    MyClass obj({-1, 2, 3});
    obj.Print();  // Output: 10 2 3    
    obj.myList()[1]=5;    /*******External modification not allowed **********/  
    return 0;
}

name 'pkg_resources' is not defined

And finally, of course we can add a setter to the C# version:

In [14]:

%use .net-csharp
using System.Collections.Generic;

public class MyClass{
    private List <int> _myList = new () {0,0,0};   // private backing store
    public IReadOnlyList<int> MyList { get => _myList.AsReadOnly(); set => _myList = new List<int>(value); } //Public property

    public MyClass(List<int> list = null){
        _myList = list ?? _myList;      
    }

    public void Print(){
        Console.WriteLine(String.Join(" ",this.MyList));
    }
}

var obj = new MyClass();
obj.MyList= new List<int>{-1,2,3}; // use the setter instead of initializer
obj.Print()  // -1,2,3

-1 2 3

And the below will fail to compile, preventing direct modification of list elements.
(Note, that values are immutable, so this is replacing the item in the list, not modifying it. Modification is still allowed.)

The .AsReadOnly() in the getter makes it run-time readonly, so prevents(at run-time) downcasts (which nobody should do anyway) to unlock it. It costs a shallow copy though and isn't required. List directly implements IReadOnlyList and can implicitly cast to it without a copy conversion.

In [9]:

%use .net-csharp
obj.MyList[2]=-1;
obj.Print();
//THIS WILL GIVE A COMPILE ERROR:

(1,1): error CS0200: Property or indexer 'IReadOnlyList<int>.this[int]' cannot be assigned to -- it is read only

We do have to explicitly declare the backing variable and getters and setters but the abbreviated syntax arguably helps if hand-writting setters, and can encourage upfront use. We only need to do this pre-emptively for lists. This syntax planned for further simplification in the next release to just:

Use IList (or combined type) for publicly-mutable C# lists

Best practice is to return read-only views and use methods to manipulate data.

But what if we really want to start by just letting users write directly to a public list without setters?

The easiest answer if you must expose direct random access is a IList from the start.

Many types can be emitted as IList simply as

public IList<int> MyList {get; set} = new Collection<int>(size);

So long as Collection or whatever type implements IList, and everything list does, even native C-syntax arrays, which are super fast.

The private variable can thus be (re)implemented as a List for performance, an Observable Collection for control, or a future custom class for internal performance and external control if needed, or whatever. Users always get a IList.

The nitty-gritty downsides

Oddly, functions that don't change data, should theoretically take IEnumberable parameters for maximum flexibility, if random access isn't required, or IReadOnlyList if it is. This let's users know they won't change data, and helps the compiler optimize. But IList doesn't implement IReadOnlyList. The prescribed way is to pass MyList.AsReadonly(), but that makes a shallow copy (slower).
Arrays internally are the fastest, but ironically they are slower than List when cast to IList. I think this is a problem in the compiler.

Solutions to Downsides

It's tempting to make a custom combined interface. I'll add a link to why this should work but doesn't well. The real solution is either

a) Just accept IList always for parameters. This is common. Just forget about using IReadOnly List. Supposedly it does produce some compiler optimizations though (which should happen anyway, but let's not get into that).

b) Accept IReadOnlyList and explicitly cast from IList. In modern code almost everything that implments IList also implements IReadOnlyList. And the opposite is even more true. To be "proper" you can put the cast in a try-catch block that falls back to .AsReadonly(). When peformance isn't critical, just use .AsReadOnly().
If you need array performance again outside the producing class, consider if it's worth copying, or do a SAFE downcast:

if ( (myarr = MyList as int[]) !=null) {
   ... stuff with myarr...
} else {
   stuff with IList
}

List is only a little faster than IList, but can check for list too. It does duplicate code though, which is annoying.

Basically C# still let's us start easy and add in checks later with one catch, we need to have been returning by IList instead of List from the start. You cannot sublcass List element access, so there's no way to change internals if you didn't start with IList.

In [4]:

%use .net-csharp

using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.Linq;


public class MyClass 
{
    public IList<int> MyList { get; set; } = new List<int> { 0, 1, 2, 3, 4 };

    // a print method. 
    public void PrintList()
    {  
        Console.WriteLine(String.Join(" ", this.MyList));
    }
}

// That's it. and we can do what we want to it.


// We can replace it and we can modify elements, anything we want.
MyClass obj = new MyClass();
obj.PrintList();
Console.WriteLine("replacement:");
obj.MyList= [-1,10,11,12,13];
obj.PrintList();
obj.MyList[2]=5;
obj.MyList[1]=-1;
obj.MyList.Add(-1);
obj.PrintList();

0 1 2 3 4
replacement:
-1 10 11 12 13
-1 -1 5 12 13 -1

Now I can do the exact same thing, but add setters to catch changes to individual elements in the list. This is a little involved. But the easy stuff (above) is easy and the hard stuff is about as easy as it could be really. We don't need to change the public

In [29]:

%use .net-csharp

using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.Linq;


public class MyClass 
{
    private ObservableCollection<int> _myList = new () {0,1,2,3,4};
    public IList<int> MyList { 
        get => (Collection<int>)_myList; 
        set { _myList = new ObservableCollection<int>(value.Select(x => (x==-1) ? 0 : x));
              _myList.CollectionChanged += MyList_CollectionChanged;                             
        }
    }

    // This is easy enough, but required research.  Simple examples are non-existant. 
    private void MyList_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
    {
        if (e.NewItems != null ){                                                            // item replacement and addition caught here.
            for (int i=0; i < e.NewItems.Count; i++){
                if ( (int)e.NewItems[i] == -1 ) {
                    Console.WriteLine($"Correcting -1 in collection element {e.NewStartingIndex+i} ");
                    _myList[e.NewStartingIndex+i] = 0;
                }
            }
        } else if (e.Action == NotifyCollectionChangedAction.Reset) {               // docs: "The contents of the collection changed dramatically."
            MyList=_myList; // Probably rare.  Just reassign whole list via setter. 
        } // other cases like removals could be caught.
    }
    public void PrintList()
    {
        Console.WriteLine(String.Join(" ", this.MyList));
    }
}


// We can replace it and we can modify elements, anything we want.
MyClass obj = new MyClass();
obj.PrintList();
Console.WriteLine("replacement:");
obj.MyList= [-1,10,11,12,13];
obj.PrintList();
obj.MyList[2]=5;
obj.MyList[1]=-1;
obj.MyList.Add(-1);
obj.PrintList();

0 1 2 3 4
replacement:
0 10 11 12 13
Correcting -1 in collection element 1 
Correcting -1 in collection element 5 
0 0 5 12 13 0

Notes on types.¶

Type compatibility:¶

It's tempting to just return a Collection or ObservableCollection, because both implement IList but then you cannot as easily or efficiently re-implement with a direct List or T[] (C-like array) internally. For public parameters, IList is generally the most flexible. For functions not meant for shared access, use whatever. IList gives you the flexibility to change implementation without breaking usage.

Possibilities.¶

It's even possible to wire in attribute changes too, but quite a bit more involved.
It's also possible to write a custom type that can use a list internally and emit an OC externally.

Can't python do this?¶

This time, yes, technically. Python collections can be subclassed and their idex modification methods can be overriden so you can make a custom container with per-instance callbacks that can be used in those methods and then class using the container can implement the callbacks.

This can be and has been written in pyhton:
https://pypi.org/project/observable-collections/ But I have no idea how trustable it is. It's not even that hard to write your own. Everything can be done, but it's real work at that point, and in an iterpretted language this probably gets very slow.

Still, none of this prevents editing attributes of objects stored in a list (without more callbacks).

Context-free element level protection¶

The next example basically works in either python or C#. It can catch changes to list item attributes. Sometimes all we care is that the objects themselves always get manipulated properly, regardless of what class they're used in. Just in case I made it an OC so we could gaurd against editing the list itself too later if needed.

In [3]:

%use .net-csharp
// ********************LIST OF NAMED ITEMS.  DON'T LET ANY LOWER CASE LETTERS GET IN!! *************

using System.Collections.ObjectModel;

public class AnItem{  // Things to put in a list. 
    string _name = "";
    public string Name { get => _name ; set => _name = value.ToUpper(); }  // Setter converts to uppercase.

    public AnItem(string name){    // ctor
        this.Name = name;          // use the public property, don't implement setter twice.
    }
}
public class myclass{   //  Class with our list of things.
  // Public list property, auto {get; set;} 
   public Collection <AnItem> MyList {get; set;} = new () {new AnItem("Kim"), new AnItem("Park") ,new AnItem("Bob")};  

   public void Print(){
       Console.WriteLine(String.Join(" ",MyList.Select(o => o.Name)));
   }
}

var obj = new myclass(); // Initialize with Kim,Park,Bob to KIM,PARK,BOB
obj.Print();  
obj.MyList[2].Name="joe";  // becomes JOE.  We trapped a modificat to an item attribute (just not context specifically).
obj.Print();
obj.MyList[2]=new AnItem("dipak");  // even this works here. 
obj.Print();

KIM PARK BOB
KIM PARK JOE
KIM PARK DIPAK

In this example we don't even need an IReadOnlyList property, since we trap all changes to AnItem.Name.

Final point on encapsulation:¶

Encapsulation is great, but immutability is becoming even more popular, and removes many concerns.

Property summary C# vs python¶

(and C++)

Bottom Line and best practice for C# and C++ data types¶

Container access control in Python is pretty lousy. C# wins here by a lot with only a small amount of planning:

C#: Return IReadonlyList of immutable, or mostly immutable objects if possible.
C++20 equivalent: probaby span view of an internal vector.
If objects must be mutable, encapsulate the objects themselves if that's sufficient (owners won't see changes though).
C#: Return IList (not IReadOnlyList) if you really feel you must provide direct mutable access without access methods. 5) If none of this is good enough, return a deep copy.

Python has no good solutions other than planning from the start to make lists entirely immutable, or returning them as deep copies.

Advanced cases: Modification of objects in Lists
If we need to (continue to) allow and now catch modification of properties of list elements, and the containing class needs to know about them (not just the contained-object class as in (1)), well, that's not impossible in C# (or even python really, just more work), but it's more work (must wire/unwire ProperyChanged callbacks from all objects added to or removed from list) and has a performance hit. This is getting very far away from slapping an @ property on anything.

Much research on Notifications came from the "Output" box here: https://github.com/LeeCampbell/RxCookbook/blob/master/Model/CollectionChange.md

My thoughts on Comparisson of C++, Python, and C#¶

It's often claimed that it's "easy" to get started writing code in python. About the only objective things I see supporting that are properites, ease of execution, and possibly some language features like list comprehension etc (but C++ is gaining these) and of course, a number of avoidable "issues" in C++.

A couple of points on C++¶

C++ has hard reputation probably because it allows a lot of old bad habits, that are built into decades of physics libraries, and related examples of horrible practices which is all many physicists ever learn from. Some of this can be cleaned up with better user practices. Some requires better physics libraries based around those practices. Some is still fundamental:

"Properties" are a pain to write, especially for people who insist on using vim (IDE's add them automatically).
The lack of named paramaters, makes it difficult to write clear constructors call, so setters may be used instead, preventing immutability.

Python advantages:¶

It normally doesn't use any pointers (raw, smart or otherwise). No deletion leaks. No memory management concerns. It uses garbage collection. This covers 90% of the issues with old C++ code for physics. This is not always preferable to or even much easier than good RAII in C++, but it's very preferable to the libraries and habits that presently are used.
It has properties!! But they don't deliver.
Integration with root and other scientific libraries.
It's obviously a scripting language. (advantage?)
It has wide availability.

C#: Isn't it just for windows:¶

Not for the last few years. it installs easily on Ubuntu and builds and edits easily in vscode. It runs with jupyter using polyglot notebooks along side other languages SoS multi-language Jupyter Notebook

But isn't C# C-something-ish?¶

Well the syntax is familiar, yes.
**Like python, it doesn't normally use pointers and does use garbage collection*** See point on python above.
Like python, it also has properties, and not perfect, but a lot better than pythons. This matters.
C# is generally easier to program in than both C++ and python.
- Python is easier to setup hello world. That's it.
- It's cleaner, more organized, and things like interfaces promote good coding, while being less intimidating and opague than "pure virtual base classes" (C# coding helps with C++)
C# is very expressive. List operations are amazing, with "LINQ" extensions removing loop operations to improve code brevity without sacrificing clarity.
Inheritance is cleanly divided into interfaces (accept anything IComparable to sort) and inheriting implementaion, with a preference for the former.
C# is faster than python (about 4.5 times faster for a PE-MC I wrote)
C# can of course run parallel threads. CPython can't.
C# does allow to make wrappers around C++ if faster code is needed (but see benchmarks below). It's some extra work but looks much easier than in python. (No C intermediary)
C# is typed, but with good generics (and now even run-time duck typing where explicitly accepted).
- Untyped code (Python) can't be error-checked. Debugging is hard.
- Type annotations and checkers exist in python but are inconsistent with unclear standards.
- C# has good generics. It seems not as powerful as C++ templates, but it usually gets the job done and is way easier.
- Typed code exposes problems earlier.
- It does have run-time type checking if really needed, through "reflection."
C# lacks all of the problems mentioned with python below.
Unlike C++,** building is easy**. Dependencies and packaging are arguably easier to manage than in python in most cases.
- No headers, no includes, much less concern with Makefiles. If classes are defined in different files, it just works! (C++20 gained modules, so this is developing).
- It is possible to create dependencies on packages stored in github LINK (true for python too).
- It is meant for an IDE and that's a GOOD thing. Modern IDE's save tons of time over vi.

C# has scientific libraries.¶

SciSharp includes ports of numpy Pandas, and much else:
https://scisharp.github.io/SciSharp/

I found Plotly.NET the best C# plotter, see C# plots and related links in SoS-install

Several HDF5 data file libraries exist (I don't know how good).

Accord.NET: http://accord-framework.net/ Is free (LGPL, not GPL, so really free) and has a Metropolis Hastings Class of unknown usefulness, along with many machine learning codes.

Other minimizers may be an issue. Minuit is not ported, but could wrapped (it has a small API). It seems like MH MCMC is the future anyway, but lack of roostats could be a bummer. ALGLIB looks interesting (MINUIT alternative) but none of these are maintained from within our field, and there's discussion this year, about getting the Hessian from ALGLIB, that seems unresolved. Chisquared is mentioned, but not likelihood.

C# problems.¶

It doesn't have unique_ptrs.
Garbage collection doesn't let you set up-front demands on memory life, and can still lead to effective leaks.
I'm seeing some issues with optimizaton of raw arrays in some situations.
Type system has some gotchas.

¶