Category Archives: Programming

Visual Studio 2017, .NET Core, MVC and EF on Mac

I’ve started playing with Visual Studio 2017 on the Mac, pulling across a sample MVC Core + Entity Framework Core tutorial project on the Microsoft site. The tutorial can be found here.

I run a Windows in a VirtualBox VM on the Mac, so I have a full Windows 10 + VS 2017 Community install with MSSQL 2016 on it.

On the Mac side, I was trying the latest VS 2017 Community for Mac, along with ASP.NET Core.

I was hoping to suss out how much cross-platform compatibility there was, and how much of a poor cousin VSMac was compared to its Windows counterpart, which is still one of my favourite IDEs.

Installing

Installing VSMac is pretty straightforward and .NET Core is a separate install documented on the Microsoft website.

Moving the project over to the Mac side

I started and finished the project on the Windows VM, running on MS Sql Server. This project opens up fine on VSMac. I had some high expectations it would, and it does. It even builds and runs!

Of course, the lack of database access on the Mac side needed to be addressed, but the web project hosted inside VSMac started up and ran just fine in Safari using localhost and a custom port number just as in the Windows VM and Edge.

Converting to use MySQL instead of SQL Server would be the next challenge.

Adding Pomelo for MySQL Support

I didn’t find too many options for MySQL. There are some hints on the MySQL blog that MySQL Connector works with Core, but I couldn’t actually find a package for the Mac listed in their downloads so I gave up. The one that looked OK is Pomelo.

Pleasantly enough, Pomelo.EntityFrameworkCore.MySql  is one of the packages listed when firing up the Project | Add NuGet Packages… option in VSMac.

Simply add the package to your project at this point and you’re almost ready to go.

Changing the Entity Framework Provider to MySQL

This was also fairly straightforward. In the Startup.cs file, the database context needs to be adjusted to use MySQL.

From:

public void ConfigureServices(IServiceCollection services)
{
  // Add framework services.
  services.AddMvc();

  services.AddDbContext<WebTestContext>(options => options.UseSqlServer(Configuration.GetConnectionString("WebTestContext")));
}

To:

public void ConfigureServices(IServiceCollection services)
{
   // Add framework services.
   services.AddMvc();

  services.AddDbContext<WebTestContext>(options => options.UseMySql(Configuration.GetConnectionString("WebTestContext")));
}

The connection string in the appsettings.json was also changed to the MySQL flavour:

 "ConnectionStrings": {
 "WebTestContext": "server=localhost;port=8889;database=thedbname;uid=myuserid;pwd=thepassword"
 }

Once this was done, running

   dotnet ef update

on the terminal command line in the project directory (where the .csproj file is located) should attach to the configured MySQL instance and create the required tables (in this case, just one) for the sample project.

And voila – things run and the database is created. Quite impressive. You can even add a new movie. But alas, not a second movie…because…

Add Auto Increment to ID field in MySQL

For some reason, the Pomelo provider, or .NET, or something somewhere doesn’t know that EF relies on the ID field on the table being an auto incrementing field. This causes any table inserts beyond the first item to fail with a MySqlException: Duplicate entry ‘0’ for key ‘PRIMARY’error.

The fix is simple enough; either:

  1. Go into phpMyAdmin and change the row property for the ID column to check the A_I (Auto_Increment) box, then save changes; or
  2. Run a SQL command to do the same thing – something along the lines of
ALTER TABLE `Movie` CHANGE `ID` `ID` INT(11) NOT NULL AUTO_INCREMENT;

Notes

  1. Entity Framework Core references must be added manually when using VSMac – you can’t add this through NuGet right now. The .csproj file must be edited manually to create the references. This appears to be a bug / limitation with just VSMac. Since I started the project on VSWin and moved to VSMac, I didn’t have this problem. But I did with a new project started on VSMac. I suspect that moving back and forth between the environments may be quite feasible.
  2. Scaffolding for helping create Insert / Update / Create views does not appear to be present for VSMac while VSWin has an option to create an MVC Controller along with associated views. These would be really handy to help build out your basic CRUD functionality. However, there may be options using Yeoman. More to come.
  3. Razor view tag helpers do not seem to provide syntax highlighting in VSMac.

Next Steps

More research to come, but the next step will be configuring the project so it runs nicely under NGINX or Apache as a Kestrel reverse proxy on the Mac without Visual Studio hosting.

 

Bitten by PHP DateTime mutability

I was finishing off phasing in new content and features of my latest project, the Surrey International Writers’ Conference (www.siwc.ca) website when some reports came in of problems with the published dates of the various writing workshops being held.

Workshops all adhere to a common schedule for each day of the 4-day conference, so I created a custom TimeSlot WordPress post type to centralize timeslots to allow sorting and grouping of workshops together on the master schedule. The data entry page for each workshop has a drop-down list box with friendly names like “Friday 10:00 AM – 11:30 PM”.

The actual format of the workshop date identifier string is “dd-hhmm-hhmm”, where dd is the day (05 = Friday, 06=Saturday, 07=Sunday, etc.) and the hhmm values specified the start and end times for the workshop. Our example workshop TimeSlot ID string as stored in the workshop metadata in WordPress would be “05-1000-1130”.

I had arbitrarily assigned Day 5 to be the Friday of the conference. This allowed the conference in future to potentially start earlier, say on Wednesday (3), or Thursday (4) to accommodate master classes or other pre-conference activities without needing to re-enter new timeslots every year (timeslots do not change year by year).

So when calculating the actual date of the workshop to display to users, I needed to subtract 5 days from the global start date of the conference (expressed as the date of the Friday), then add back the number of days in the workshop date structure. For example, for a Saturday 10:00 AM – 11:30 AM workshop, the ID would be (06-1000-1130).  Taking 5 days away from date of the Friday (October 20, 2017), then adding 6 days gives us October 21, 2017 as the date of the Saturday.

That’s where the problem arose…

Here’s the original code, taking 5 off the conference start date, $siwc_conference_start (DateTime object), then adding the workshop day value that I extracted previously from the string.

// $day = workshop day, 5=Friday, calculated previously

// following line modifies $siwc_conference_start!!!

$thisdate = $siwc_conference_start->sub(new DateInterval('P5D'));  

$thisdate = $thisdate->add(new DateInterval('P' . $day . 'D'));   

return $thisdate;

This caused no end of problems! I was getting seemingly random dates for the Friday – like October 23, October 21, etc.

After much tracing, I isolated it to the fact that the all-important $siwc_conference_start variable, which should have been, and needed to be, constant, was actually changing in value between calls! The only possibility was the sub() method call (and the add(), of course, upon later reflection) affecting the object directly rather than simply returning the result of the calculation.

This outlined an issue I hadn’t been aware of in PHP regarding the mutability of the DateTime class, which $siwc_conference_start was declared as.

The sub() method was actually acting upon the $siwc_conference_start object, subtracting 5 days from the conference, rather than returning an object with the 5 days subtracted from it and leaving the original alone.

The subsequent add() method was adding a different value back to the conference start date. So the conference start date value was bouncing between different values, some actually correct, depending on prior calls. This also explained why I hadn’t caught the bug in my earlier tests – my test data didn’t exercise the function sufficiently (either it had all Friday data, or I didn’t notice the discrepancy).

The  fixed code:

// $day = workshop day, 5=Friday, calculated previously

$thisdate = clone $siwc_conference_start;
$thisdate->sub(new DateInterval('P5D'));
$thisdate->add(new DateInterval('P' . $day . 'D'));

return $thisdate;

This is one fix – using the clone keyword to create a new object containing a shallow copy of the $siwc_conference_start object’s contents. The sub() and add() methods will act on that copy, not affecting the original.

There is also a second fix: declaring $siwc_conference_start as a DateTimeImmutable class introduced in PHP 5.5. This new class is an admission that the original DateTime class implementation was flawed in allowing the original object to be changed. The new implementation in DateTimeImmutable returns a copy of the object as would be expected.

Overall, this highlights a couple of interesting things around object mutability, when it is expected, and when it is not. The original implementation breaks a couple of different programming language expectations…

  1. Caused by my having used other languages’ DateTime types. For example, the .NET DateTime is a value type, therefore would behave as expected in the first implementation.
  2. Caused by expectations that DateTime should be implemented best as a value object rather than reference object. There are few situations where DateTimes are likely to be better off as reference objects (e.g. objects meant to be shared) than value objects. It’s not that the implementation was technically wrong, but the expected usage by programmers did not align well with the initial design decisions for this class.

This is also a lesson for creating a proper test harness for functions like this that are easy to test. I could easily have created a sample set of timeslot IDs and corresponding results, and this would have been identified before going into production.

Javascript or JQuery – which first?

I’ve been asked this quite a number of times recently – should a web developer learn JQuery or Javascript first?  And presumably pick up the other later.

This question would likely apply to other Javascript libraries or frameworks (e.g. Bootstrap).

In other words, does starting with the fun and useful application of Javascript (JQuery) and seeing web pages come to life help to ease one into the drudgery of learning to program?

I have to admit that I’m a programmer first, thus I would fall into the Javascript first camp somewhat by default. My immediate reaction would therefore have been, “Javascript first, duh.” But I wanted to think it over.

My tendency is to learn the hard, foundational stuff first, then move on to the easier stuff later. It’s a bit like learning to do long division on paper, then moving on to use a calculator.

On the flip side, I admit using a ton of JQuery plug-ins way before I learned to use JQuery itself or even got deep into Javascript. This was driven by the desire to find the absolute best image slider for my ASP.NET website without having to do any coding. So I know there will be people that can quite cheerily create a really nice image slider on a webpage using a few lines of HTML without knowing any programming whatsoever, and it’s fantastic that they can do this, and often for free or very little money.

However, if you want to go beyond simple plug-ins on a page, you rapidly run into needing Javascript knowledge:

  • Any really useful functionality will still be written in Javascript; you need to declare variables, use loops and other constructs
  • JQuery IS Javascript, it’s not an either/or, it just is a simplification of what you can do in Javascript. It lets you code less to do the same thing, which is the hallmark of a good framework
  • Some of JQuery’s syntax of passing functions as parameters and nesting of functions within functions or objects can look very intimidating visually (and tough to unravel mentally) compared to starting with Javascript first
  • Knowledge of things like events, event listeners, the DOM, etc. are still required in order for a page to do something useful
  • Understanding how something works at a basic level I feel is crucial so that you know how a framework simplifies it for you (the long division argument above) – for example, knowing that a JQuery function might be implemented as 3-4 lines of Javascript code takes some of the mystery out of JQuery
  • There are many other frameworks based on Javascript, so learning the root Javascript language will help you understand how these frameworks work
  • Debugging is facilitated by knowing the underlying Javascript code – sometimes an error will occur in the JQuery code that you’ll have to figure out why.

So, for budding web designers, I’m firmly in the bottom-up “learn some programming basics first” camp, which can be done conveniently with Javascript, and then learn the simplifications later. I was also supported in this view by hearing responses to my question, “Did you actually know what you were typing, or were you just guessing and fiddling with your JQuery code until you got it to work?”

There can be a dividing line – learn programming basics, then move to DOM events and listeners, manipulating DOM elements, and once you’re comfortable writing a couple of simple interactive pages, dive into JQuery and see how those same things can be simplified substantially. You’ll understand those magic incantations better.

The only real exception is if you are going to strictly use the framework to do some of the fun UI plug-in stuff – if you can figure out how to tweak your HTML to add a JQuery plug-in, go right ahead and good luck!

Beginning Coding Tips

Since I’ve been involved in coaching at BCIT, I’ve started to get a better insight into the learning processes and tools required to get a grasp of ‘programming stuff’.

Programming isn’t easy by any stretch, so I may not be able to nail exactly what it is that may be that single ‘aha’ moment of clarity that makes programming all easy from that point on, or if there are required to be many of these along the road. Or if there is a road, or a meandering goat path, or a highway.

I’ll begin with the basics you have to understand. I won’t rehash a lot of what should be covered, just various observations and clarifications.

Basics

Variables

OK, so maybe not that interesting, but the whole concept of variables, their being at the very simplistic level a box that holds something in memory.

Understand naming conventions – does your language care about upper and lower case when naming variables? Javascript does, and is probably the cause of the greatest frustration with all levels of coders. Just be aware that if something isn’t working, this could be the simple reason.

Variable Types

Variables have a type associated with them — this is a bit of a peek behind the curtain into how they might be stored internally. Usually a variable will either be a number, a string, or an object.

A number is basically something you can conceptually punch into a calculator  to do some math with and it’ll make sense. Historically, computers have liked to store numbers in specialized formats so you may find in other languages different types that represent integers (whole numbers only) and floating point (all numbers, including fractions) and your having to choose the right type for what you’re doing.

A string is literally just text, aka a string of characters you can type on your keyboard (and many more you can’t easily type) – alphabet, numbers, emoji, characters from the world’s languages, etc. Strings are quoted with single or double quotes.

An object variable is, well, an object (see objects later).

Automatic Type Conversion

Types can be converted from one to another if it makes sense – the string “123.45” can be converted into the number 123.45. But the string “A123.45” can’t be converted.

Javascript attempts to do automatic type conversion when it makes sense; but beware of the rules; i.e. the following weirdness:

"3" * "2"    // this returns the number 6
"5" + 1      // returns the string "51"
"5" - 1      // returns number 4

You can see that it’s beneficial to explicitly tell Javascript what you want done instead of relying on the automatic type conversion, which you can do by calling the Number() and String() functions:

Number("3") * Number("2")   // returns 6
"5" + String(1)             // returns "51"
Number("5") - 1             // returns 4

Variable Scope

Understanding when a variable is ‘alive’ and valid is a huge concept. Most current languages allow the same variable name to be declared again within a different lexical scope; i.e. inside a function as well as in the surrounding code. These are logically two different variables, so do not get confused.

In this Javascript example, the variable ‘a’ is declared twice; within the function, and outside the function. The inner ‘var’ makes a whole new local variable ‘a’ that is independent of the outer ‘a’ which has a global scope.

var a=3;      // global

function f()
{
  var a = 5;   // independent (local) variable
}

function g()
{
    // in here, you can access global 'a'
}

Whereas in the following example, the inner ‘a’ actually refers to the outer ‘a’, which may or may not be intentional.

var a=3;      // global

function f()
{
  a = 5;      // affects the global a
}

Quirks abound in Javascript due to its automatic variable declaration:

function f()
{
  a = 5;      // creates a global variable a !!!
}

Having functions ‘reach outside’ their scope is usually a bad practice and can lead to unintentional side-effects.

Best Practice:  Always declare variables with ‘var’ instead of allowing Javascript to figure out if you’re using a new variable or an existing one.

PHP Note: Of course, PHP has to different. PHP scopes variables only to the scope they are declared in. This is a very frequent pitfall for PHP programmers used to other languages. Global variables look like they’re declared, yet they’re unassigned when you try to use them.

Access to a global variable in an outer scope has to be explicitly declared. Why? probably to protect you from accidentally changing a global variable. So to access a global from a function, you need the global keyword like so:

$g = 5     // a global variable
function addG($num)
{
  global $g;    // have to have 'global' here..
  return $g + $num;
}

Functions

Understanding that functions are ideally written without any reliance on external knowledge of who’s calling the function. In standard terms, this is to reduce side-effects as noted above.

Functions written this way can also be modular; think about giving someone else this function to use in their code – will it be possible, or will you have to do something to their code or yours in order to make it work properly?

Parameter Passing

Understanding that parameters in a function are placeholders within that function and allow you to specify functionality without needing to know anything about who’s calling the function.

This again is an example of scope – function parameters are simply just scoped to the function.

Objects

Understanding the concept of objects is essential in working with the HTML DOM and pretty much needed to anything useful such as storing/retrieving data. So much functionality is wrapped inside objects, or arrays of objects, and so forth.

Understand that just about everything has been ‘object-ified’ — it’s one way of looking at the world in programming terms of objects, functions, fields, and events.

Arrays / Collections

Objects become more useful when you can have more than one of them (money, for example, or students, or houses).

Understand moving through collections of objects one at a time, or directly through key values. You’ll do this often; i.e. grab a list of student objects and display all names on the screen.

More advanced concepts

Dig into HTTP

Understand a little bit about the mechanics behind web traffic – GETs, POSTs, cookies, and so on, so that you know that a server is involved somewhere – not everything useful executes purely in the browser.

This positions you well to use the raft of services available through REST web services, etc., and inevitably to try to debug some of your applications that use web services.

Object Models

An Object Model, viewed through Javascript googles is a bunch of related objects, with properties and functions to manipulate those functions. An OM tries to model something in the real or abstract world in some sort of logical way so that you can do useful things with it programmatically. An Object Model for a car might look like this:

var car = {
  make: "Mazda",
  model: "CX-5",
  radioPresets: [ "104.5", "96.1", "89.3" ],
  startTheCar: function() {},
  driveTo: function(location) {},
  pickUp: function(persons) {},
  setRadioToPreset: function(presetNo) {}
}

So a car object becomes a handy way to encapsulate in one handy package useful information about an instance of a car (“properties” such as make and model) and to also encapsulate the useful functions a car can perform (driveTo a location), otherwise known as “methods”.

The packaging aspect is a convenience in that you can call methods and access properties using the “.” operator; i.e.

car.make = "Volvo";
car.pickUp(["Martin", "Kathy"]);
car.driveTo("Home");
car.setRadioToPreset(1);

You can hopefully envision a world where objects interact with other objects (cars interacting with road objects), objects containing other objects (cars containing arrays of people objects to represent their occupants), objects composed from other smaller objects (car containing an engine object, a radio object, 4 wheels objects), each composed of sub-objects.

Aside for sticklers: the code above is really for illustrative purposes; current Javascript provides a way of defining objects through functions and prototypes in a more object-oriented way (i.e. compared to something like Java or C#).

HTML DOM

More on DOM stuff later; for now, understand that this is an object model (hence the “OM” in “DOM”) that represents in a tree-like form what you see in a browser. You can manipulate the objects in the tree (setting properties and calling methods) to control the visuals in the browser window.

Javascript braces hell

Start to understand Javascript’s approach of functions as parameters to functions, and the resulting nested curly braces. Format your source code properly to foster readability

Events

Understand the concept of events as notifications that occur due to some user or system action.  Remember that everything is an object, and objects can trigger events when something happens to them – like when a page is loaded, or when an image notices that a mouse pointer has moved into its field of view.

Frameworks

Finally, we get to Frameworks. Frameworks are just pre-written code, not magic. The advantage is building stuff faster, likely more bug-free, and more functionality than doing it yourself. The downside is frameworks sometimes don’t do exactly what you want, and you can easily get into a kid in a candy store situation of grabbing this UI gadget that uses such-and-such framework, and another one using a different framework — with the downside of bloating your page and making it slow or buggy if the frameworks don’t play nice with each other.

But they sure save a lot of time, so just be aware and do your homework.

Philosophy

Reusability and Modularity

I mentioned one tip earlier – write your code in a way that someone might be able to re-use something you wrote without really needing to modify it much, or at all. A simple rule

If you have a simple function, regardless of how trivial, think about whether it could be refined in a way so that someone could either include the piece of script, or include the function (copy/paste) and use it right away. If not, then it means that there is some dependency on something that exists that shouldn’t be there — probably on a variable you have in your code.

Continuous Improvement – Refactoring

It’s rare for a beginner programmer to understand how best to break code into reusable and modular bits.

On one hand, if you don’t break down a module into potentially reusable pieces, then you end up having a huge, monolithic piece of code that looks like it has a lot of repetitive stuff in it.

At the other end of the spectrum, you might break something down into much too many small pieces that, while modular, may seem excessively fussy, and may make code readability suffer.

What to do? I typically notice code that I’ve written a few times; say,

document.getElementbyID("welcome").innerHTML = "Hi There";
document.getElementbyID("status").innerHTML = "123";
document.getElementbyID("status").innerHTML = "456";

And think…maybe I can make a function with parameters to do the same thing, and name it something more descriptive. So I refactor my code a little:

function DisplayText(element, text)
{
   document.getElementbyID(element).innerHTML = text;
}

DisplayText("welcome", "Hi There");
DisplayText("status", "123"); DisplayText("status", "456");

Better! This illustrates a simple way to cut down typing at least. And you can go one further:

function DisplayText(element, text)
{
   document.getElementbyID(element).innerHTML = text;
}

function DisplayStatus(statusText)
{
   DisplayText("status", statusText);
}
DisplayText("welcome", "Hi There");
DisplayStatus("123");
DisplayStatus("456");

And so forth. Is this better than just the straight document.getElementByID method? Not necessarily, as perhaps readability is impacted (someone will have to check to see what DisplayStatus() does the first time they see it).

But what happens if you want to change where the status message goes? Then you just go to one place, DisplayStatus() instead of the two (or more) places in the code you would need to if you used the document.getElementbyID() technique.

All I can really advise around modularity or creating functions is to be alert for

  1. repeated code – that may be a candidate for creating functions to do stuff and to minimize cut/paste errors
  2. useful functionality – maybe I need to put “, ” in between two strings to format a name (“Doe, John”). That might be useful elsewhere, so a function would be a great idea.
  3. centralized functionality – things I know I may want to change often or later – like DisplayStatus() above. Maybe I want to make the message appear in a red colour or different font. Making it centralized ensures that when I change the function, all possible places that I display the status also change with it.
  4. deferment – should I just toss in a function as a placeholder for now, and get on with the rest of the coding that I have in my head, and get back to that later?  I.e.  BigMathFunction(x, y, z, a, b)
  5. plain old readability. Can I just chunk up a huge function into smaller functions and call them in the same sequence; i.e. BeginningPart(), MiddlePart(), EndPart()