How to type an end-of-line

Recently a fellow programmer asked me a couple questions about how to type a linefeed on the keyboard, and how to handle newline delimiters from one program to another.

1. "What keyboard key can I press to create a linefeed character on a Mac?"

Depends on the app. End-of-line delimiters (also known as Newline) are not really input by the keyboard. The app interprets a press of the Return or Enter key, inserting into the data what it believes to be appropriate.

In most text editors such as TextMate, JEditBBEdit, TextWrangler, etc. you specify in prefs what the end of character should be. Some setting like "Lines end in: " with a popup menu offering LineFeed, Carriage Return, both. An app might even offer some special Unicode characters meant explicitly for such use (that nobody uses unfortunately). 

So, the upshot is: The user presses Return (or whatever) and the app inserts the particular octet(s) for a LineFeed, Carriage Return, both, etc.

2. "I was having some issues with 4D text fields that were getting pasted in HTML code from TextMate which was supposed to be putting in linefeeds."

Yes, sharing data between apps is almost always tricky. The programmer or user must set/manipulate the desired end-of-line delimiters either in the data source (TextMate here) or the sink (4D here).

Remember that computers are illiterate. Ultimately they know nothing about characters and text. They truly only understand numbers. What you think of as a file full of text is actually a file full of numbers. If an app reads that file and encounters a "10" (Linefeed), the app may choose to treat that octet as an end-of-line. Or that app may ignore each occurrence of 10, blissfully cruising along looking for an expected '13' (Carriage Return) on which to break lines. To really see what an app sees when it opens a file, to see octet-by-octet, open the file yourself using a hex editor such as HexFiend.

Believe it or not, the geek in me finds this kind of topic fascinating: human language vs computerized data, character sets, Unicode, etc. I have actually had a friend call me to discuss such topics as a cure for his insomnia. Don't even get me started on the lack of foresight of the early computer information folks in the `60s and `70s.

  • Many text editors do not convert existing documents when you change the end-of-line prefs setting. Your setting might apply only to future new files. 
  • You can verify what is being used with a hex editor such as HexFiend.
  • Some older Mac programs still default to Carriage Return for end-of-line rather than the Unix convention of linefeed.
  • Some cross-platform development tools try to help by defaulting to the convention used on the computer at runtime. But this can cause confusion as it changes the behavior of your app depending on which computer a user uses. Usually you should hard-code a specific consistent end-of-line. Java and REALbasic switch defaults, and as I recall 4D does too at least in some circumstances.
Here's how to hard-code generating a linefeed in REALbasic:


Nil versus Null

The 4D language has two words that may be confused: Nil & Null.

Nil tests whether the pointer points to a valid container of a value, a field, variable or array. Nil has nothing to do with the content of what is being pointed to. Nil tells you nothing about what may or may not be inside that field, variable, or array.

  • To test if a pointer is nil:
    If ( Nil( $myPointer) )
    End If

NULL means the container (a database field) has no contents. No value = NULL. Do not confuse no value with an empty value. For an Alpha field or Text variable, we represent an empty value in our code as a pair of double-quotes: $companyName := ""
  • To test for NULL, call the command Is field value Null
  • To set a NULL, removing the value entirely, call the command SET FIELD VALUE NULL

Here's code to test for nil pointer…

C_pointer( $pointerToText )
if Nil( $pointerToText )
  `ERROR condition.
else ` else not nil, we have a valid pointer. So use it.
  C_TEXT( $myText )
  $myText := $pointerToText-> `Copy the value of the text.
end if

You should always test a pointer for nil when passed as a parameter, as seen in this code…

if Not(Nil($1))
end if

4D versions 11 & 12 introduce the feature/problem of NULL values in a record's field, as do most other SQL databases. You can have a valid pointer to a record's field, but have no value (NULL) stored in that field. On the other hand, the record's field may have a value (be non-NULL) but that value may be empty such as empty text for Alpha Field, !00/00/00! for date, etc.

So generally you should be testing for 2 or 3 issues:

  • Is pointer valid? (not Nil)
  • Is what's being pointed to a record field with a value? (not NULL)
  • Is the value valid in the record field being pointed to? (not empty, within expected range, etc.)

--- 4D code --------
C_pointer( $pointerToField )
C_TEXT( $myText )
$myText := "ERROR - This text never got set"

if Nil($ pointerToField )
  `ERROR condition.
else ` else not nil, we have a valid pointer. So use it.
  IF ( Type($pointerToField->) = Is Alpha Field )
    If ( Is field value Null( $pointerToField )
        `ERROR condition. No value (NULL).
    Else // Else we have an actual value

      If ( $pointerToField-> = "" )
        $myText := "Got text? Nope." `Flag error condition if having an empty string was unexpected.
        $myText := $pointerToField-> `Copy the value of the field.   <-------- GOAL
      End If

    End IF

    `ERROR - Expected an Alpha field, but got a pointer pointing to something else.
  End if
end if
… Do something with $myText

If you're working with Date, Blob, or other types of variabless or fields rather than Alpha/String, same concepts apply. Just change the data types in code above, but logic is the same.

Caveat: These code examples were written freehand, not written or tested in 4D.

Tip: In general you should avoid dealing with NULL by defining your database fields as "NOT NULL" in SQL, meaning a record with a NULL in that field will be rejected by the database engine. Chris Date explains why.


REALbasic code to connect to Postgres

I have another blog post with example code in REAL Studio for connecting with REAL Server.

Here's the same example adjusted for connecting to a Postgres server.

  dim db as PostgreSQLDatabase = new PostgreSQLDatabase

  // ---------------- Prepare connection.
  db.Host = ""
  db.port = 5432 // 5432 is default port for PostgreSQL.
  //db.Encryption = 128 // Always encrypt your connection when on an untrusted network.
  db.UserName = "postgres" // "postgres" is the default admin user name.
  db.Password = "YourPostgresPasswordGoesHere"
  db.DatabaseName = "YourDatabaseNameGoesHere" // Postgres supports multiple databases. Which one do you want? Default is name the same name as user, such as 'postgres'.

  // ---------------- Try connecting.
  dim status as String
  if( db.Connect ) then
    status = "Success connecting to db server"
  else // Error connecting
    status = "Error occurred when connecting to database server. Code: " + str(db.ErrorCode) + " " + db.ErrorMessage
  end if

  // ---------------- Execute a "SELECT *"
  dim sql as String = "SELECT * FROM YourTableNameGoesHere"
  dim rs as RecordSet = db.SQLSelect( sql)
  if(db.Error) then
    status = "Error when running a SELECT. Code: " + str(db.ErrorCode) + " " + db.ErrorMessage
  else // Else normal.
    if( rs = nil ) then
      status = "ERROR - RecordSet is nil"
      status = "Record count = " + str( rs.RecordCount ) // Replace with more useful code.
    end if
  end if


  // ---------------- Catch any exception thrown by code above.
  if( db <> nil) then
  end if


Hiding extraneous stuff in pgAdmin3

When starting out in Postgres 9 using the bundled pgAdmin3 app, I was confused by all the various pieces of the installation. Postgres has many internal support files, tables, and such, none of which is useful to a newbie like me.

Thanks to SeaPug, I got a tip to hide all that extra stuff, instead showing just my own tables, etc.

  1. In Preferences, choose the Browser tab, and uncheck "Show System Objects". 
  2. Restart pgAdmin3.


Port Number Problems

Someone complains: "Port 8080 is FUBAR for some reason..."

What do you do? Here's a few things to try.

Sanity Check

Try another computer, another OS, another human, another Ethernet cable, etc., as a sanity check.

Point Your Web Browser

Check that no other app is listening on the port in question. Without running the system you are trying to reach, point your web browser to the port in question, such as:


If the target app is running on your own computer, use the "localhost" IP number that always means "here on this computer locally":

After a pause, the web browser should report an error saying it could not connect. If it does seem to connect, you may have an app already listening on that port.

Port Scan

On a Mac, another way to check for other apps using that port is running the app:
Applications > Utilities > Network Utility > Port Scan

Chere are over 64,000 ports, so you may want to narrow the range to go faster. If a port is in use, it is listed. Not being listed means the port is free at the moment.


Check your firewall(s).

Mac OS X 10.5 & 10.6 have two firewalls in place, acting in parallel. 

• One firewall tracks which applications have permission to receive incoming network connections. This tool is turned off and on in System Preferences in later versions of Mac OS X. 

• The other firewall inspects network packets as they flow in and out of your computer. Any number of rules can be created to allow, deny, or modify those packets. 

In earlier versions of Mac OS X, you controlled this firewall in System Preferences, but in later versions this firewall is invisible. Use the command line tool 'ipfw' or download the GUI wrapper around ipfw, an app called "WaterRoof". 

Read my blog post for more info about 'ipfw'. 


If all else fails, reboot your computer, cold. Seriously.

Sniff Packets

Use the "tcpdump" command line tool in Mac OS X (and other Unix-based systems) to look at your network packets. Apple tells you how. Use WireShark or other such GUI tools to do the same. Not in my skill set, but perhaps in yours or a colleague’s.


OmniGraffle - Export Current Selection

I just discovered a super useful feature in OmniGraffle Pro version 5.2.3:
File > Export > Export area > Current Selection

I routinely use OmniGraffle to decorate screenshots with call-outs, or create graphics to demonstrate visually. But I often used other programs to finalize the image file. Now I have discovered a shortcut within OmniGraffle:

  1. Select the items I want in my image file.
  2. Choose File > Export.
  3. In the dialog, set "Export Area" popup to "Current Selection".
  4. Name the file and save.
By the way, another shortcut I've discovered is saving directly to my iDisk rather than to my local hard drive, to move files to my web server. iDisk can be flaky, so you always run a risk of problems, but lately in Mac OS X 10.6.4 I have found OmniGraffle, Preview, and other apps to work well saving a file directly.


EndOfLine in REALbasic

REAL Studio 2010 offers the command "EndOfLine" to generate a platform-savvy sequence of one or two characters commonly used to mark the end of a line of text or other use as a delimiter.

But we don't always want a platform savvy choice. Using that choice means our software runs inconsistently from one host OS to another.

To generate a specific end of line character(s) in REALbasic type:
  • EndOfLine.Unix 
  • EndOfLine.Windows
  • EndOfLine.Macintosh
The .Unix choice generates a LineFeed, ASCII/Unicode 10.

The .Windows choice generates 2 characters, a Carriage Return and LineFeed, ASCII/Unicode 13 and 10.

The .Macintosh choice generates a Carriage Return, ASCII/Unicode 13.

That .Macintosh choice is old-school, common in the days of the classic Mac OS 8 and 9, before Mac OS X. By contrast, Mac OS X is Unix-based. While many older Mac programs may still expect lines to be delimited by a carriage return, newer programs expect a linefeed. 

My own default on any platform is the .Unix choice, a linefeed, unless I have some specific reason otherwise.


Installing Postgres

I am trying the release candidate for PostgreSQL (more commonly known as Postgres) 9.0, on my Mac. [Update: Now using 9.0.1.] Version 8.4.4 is current, but given its imminent release I thought I'd give 9 a whirl.

Know that 9.0 is not a radical revision. Original plans labelled it 8.5. Some major features are being added, making it a big enough change that they decided to call it 9.0. But apparently the core architecture is the same. The bundled REAL Studio driver ("PostgreSQL Plugin.rbx") has not been updated for v9, but is working well for me so far as of R.S. 2010 Release 4.1.

Caveat: This discussion is focused on Mac OS X. Some details may vary for other platforms.
Caveat 2: I'm no expert on this topic. Take with a grain of salt. I'm just sharing my current understanding.

Where to Download

Postgres is an open-source project with its own web site.  Some commercial companies are closely involved, employing key developers and providing technical support and other services. One of these companies, EnterpriseDB provides some of the binary (compiled and ready to install) builds. So, for the Mac OS X release, you go to the Postgres.org downloads page offering 3 different download options, including a download link that takes you to EnterpriseDB. That link was for the "One click installer" option which I chose rather than the Fink or MacPorts option.


A "cluster" in Postgres can mean either of two totally different things:

  • Multiple computers cooperating in a way as to provide fail-over or other such features.
    - This is the meaning used commonly in most computing topics.
  • A single installation of Postgres on a single computer.
    - This is the usual meaning in a Postgres discussion. 
I have no idea how the term "cluster" came to be used this way in Postgres history. But get used to it. If you simply download the Postgres installer, run it once on a single Mac, you have a "Postgres cluster". The meaning is singular in the sense that you have one copy of the software installed, but plural in the sense that Postgres supports multiple database simultaneously running. Your Postgres cluster may have one database listing all the books and music you own, while also running another database with your sports league statistics. The data is stored in separate files on disk, but are both managed by Postgres simultaneously.

Computer > Cluster > Database > Schema > Table > Columns > Rows

A Mac has a Postgres cluster installed. That cluster will have at least one database (named 'postgres' to match the user account created when Postgres is installed), and you can create additional databases. In each database you have at least one schema named 'public'. Generally you can do all your work in that schema. Then you define one or more tables. Each table has one or more columns. After that you can create the actual records in the database, the "rows".

Installation Guide

Preview the steps taken during installation by reading this thorough Installation Guide. There is one apparent error: A screenshot shows port 5433, but the default is actually 5432. You can skip reading the Non-interactive installation section. However, do pay attention to the Uninstallation section at the end, as you cannot just toss Postgres in the Trash like a common app.

Specify UTF-8

From what I can gather, generally speaking, we be using UTF-8 as the character encoding used to store our data in Postgres. You can specify the character encoding for each database individually. But more convenient is specifying the encoding for the Postgres cluster, which then becomes the default for any databases created later.

You specify the cluster's character encoding when installing Postgres. I'm not sure, but you may not be able to change this later. The default is mysterious as it is platform-specific. I've been advised to override this to choose a specific encoding. This is done by choosing one of the "locales" with "UTF8" in its name. Apparently Postgres is unfortunately combining two ideas that should really be handled separately:

  • Locale - What rules should be followed for sorting, interpreting dates, etc.?
  • Character Encoding - How should character codepoint numbers are allowed, what letters do they represent, and how should they be written to disk?

For work here in the United States, I want the locale rules usually followed for US English. And I want character encoding to be UTF-8.  So I choose "en_US.UTF-8". This "Locale" setting is the only one I alter during installation; For all others I go with the default.

"postgres" User

The installer ran well without incident on my Mac OS X 10.6.4 (Snow Leopoard) MacBook. I went with all the defaults. Installer runs for several minutes. Your system password is required, for an unusual purpose: To create another Unix user on your Mac. A user named "postgres" is created, and you'll be prompted to invent a password for this user. Be sure to write that password down. That is the password for utilities to connect to the database installation, but is also that new user's password.

The main place for the installation is /Library/PostgreSQL/. Nested is a "9.0" folder, inside of which you'll find a "data" folder. Note that you do not have access to that "data" folder; it is owned by the "postgres" user. Postgres is one of the most secure database systems available. The blocked folder is one aspect of that security. If any of the other user accounts is compromised, rogue software will not have direct access to the database files.

So how do we access the database from our usual user account if we can't get to the database directly? We use utility programs, such as "pgAdmin" to connect to the database using password credentials. For example, to backup the data from the database, you can use "pg_dump" to extract the entire database definition and data as SQL script files.


After the installer completes, the "StackBuilder" utility runs. This tool is a convenience for downloading updated or additional pieces to your Postgres installation. Using StackBuilder is optional, and you can skip it for your first time usage.

Notice the "PostgreSQL 9.0"  folder in /Applications. You'll find the admin tool "pgAdmin III.app" and a "Documentation" folder.

Shared Memory(?!)

Screenshot of Mac OS X installer disk image (DMG)
Last came one confusing speed bump in my installation experience: 'shared memory'. While the issue is confusing, don't be scared… It may turn out to be a non-issue for you as it was for me.

Next to the installer on the DMG is a README. The document is short, with a strange and confusing discussion about critical "shared memory" settings in BSD systems, which includes Mac OS X. It talks about memory amounts for settings with names like shmmax, shmmin, shmmni, shmseg, and shmall. Also read the linked web page.

You needn't concern yourself with these settings just yet. The one-click installer for Postgres 9.0.1 automatically creates a "sysctl.conf" file in the folder "etc" folder at the root of your boot drive:
You must restart your Mac for those settings to take effect.

The most important thing those settings do is raise the limit on the amount of memory used by Postgres from 4 megs to 33 megs. That is sufficient as a beginner starting out with Postgres. But is still a skimpy amount of memory for Postgres to operate. As your databases grow, and for deployment to a production server, you must re-visit this topic and tweak that "systctl.conf" file. When that time comes, see my blog entry and the Postgres doc.

That concludes the installation portion of my Postgres experience. Postgres should automatically be running at this point. In the Activity Monitor program, see several processes named 'postgres' and owned by the 'postgres' user. Launch the newly installed "pgAdmin" app installed in your Applications folder. With pgAdmin, you can create a new database, tables, columns, and rows.


FastCGI with Java Servlet containers/servers.

As the exciting debut of REAL Studio Web Edition arrives, the question of deployment arises.

You have two approaches:
  • Quick and simple: Compile with an embedded HTTP Server.
  • Integrate with another web server: Your app is called by the web server via the FastCGI protocol.
The pros and cons: The built-in server is quick and easy to deploy, while fronting with a separate web server allows scalability, sophisticated admin, encryption, and more.


If you need encryption to to the end-user, then you must use a separate web server.  The built-in web server is not capable of supporting SSL/TLS encryption. To protect the communication to the end-user, you must use a separate web server with SSL/TLS features.


For those choosing the separate web server approach, discussion amongst the Web Edition crowd currently revolves around using the Apache HTTP Server ('httpd') as the front-end, calling your Web Edition app. But 'httpd' can be a bear. Some of us would rather use other web servers. For years, I have used Apache Tomcat as a standalone web server as well as for hosting Java Servlets.

Fortunately, there seems to be a good Java Servlet, jFastCGI,  available to enable FastCGI on Servlet-enabled web servers such as Apache Tomcat, Jetty, Oracle Glassfish, JBoss, and others. People seem to be mainly using this to combine a PHP server with their Java server. But it seems possible that RS Web Edition may work just as well, using a couple elements in a simple XML config file.

I've not yet tried jFastCGI. If you have any experience with this servlet or others, please post here. I tried jFastCGI, and got it working. At least the basics are working. Read this other post.


The Problem of Port 80

For those new to web serving on Mac OS X or other Unix-based computers, you face the problem of port 80.

The issue exists on Unix systems (Mac OS X, BSD, Linux, Solaris, etc.), but not MS Windows computers. By tradition port numbers 0-1024 are restricted to apps running as superuser, as a security precaution. Superuser can do anything, including wipe the entire boot drive. So normally we don't run apps as superuser. The default port for http (web servers) is 80, and is in that range. So we have a Catch-22.

A common solution is to run a web server on a port number higher than 1024, often 8080 or 8888. Those numbers are arbitrary, chosen for their cuteness. Then you can use the "ipfw" firewall built into BSD-based systems such as Mac OS X. Linux and other Unix systems have similar firewall tools. The firewall tool can do "packet forwarding" where an incoming packet is inspected and then blocked, allowed, or modified. By modifying, the packet's port number can be changed from 80 to 8080 (or whatever you chose). Then the packet is sent on its way, to arrive at your web server running on port 8080 (or whatever you chose).

'ipfw' is a command-line tool. It works by submitting each incoming or outgoing packet to a list of rules. The first rule whose criteria matches that particular packet is applied, and no further rules are checked. You can have up to about 64,000 rules.

The syntax for these rules is arcane, and not intuitive. Fortunately, I give you the very rule you need below.

Obviously, this 'ipfw' tool is critical. So you must access it with "sudo" before it, to run the tool as superuser. Mac OS X' Terminal app will prompt you for your system password before continuing.

To see your rules, in Terminal type:
sudo ipfw show

To add the port-forwarding rule, in Terminal type:
sudo ipfw add 100 fwd,8080 tcp from any to any 80 in

To delete all your rules, and reset to default, in Terminal type:
sudo ipfw flush

You need prepend "sudo" only once. It stays in effect for at least several minutes.

That's easy enough. If you wish, you can download a free gui app a gentleman created for the Mac-using 4D community, Simple Port Forwarder. That app does one simple thing: add the rule I listed above.

For another free gui tool that does more 'ipfw' work, you may download WaterRoof.

Keep in mind that for both those apps, you are entrusting your system password to their creators. Do so at your own risk.

I have done this port-forwarding with complete success when using both 4D and Apache Tomcat web server (a Java-based web server and Servlet engine, not to be confused with the "Apache Web Server").

Mac OS X always has at least one rule:
Allow all packets through regardless. 
A security-conscious admin may choose to add a rule before that:
Deny all packets. 
Then she will add more rules before that, to let through only the kinds of packets she expects. As a newbie, you don't need to think about that, but you may want to get there eventually.

Some people mistakenly perceive this packet-forwarding as a trick, a hack, or a burden to the computer. Packet-forwarding is none of those. It is completely "normal". 'ipfw' is built into the core of Mac OS X. Understand that ipfw has always been running on your Mac, already inspecting every packet coming in or going out. You are simply adding one more rule to be considered and applied.

The info above relates to Mac OS X and other BSD-derived systems. Linux is not BSD-derived, but does have a similar packet-inspecting firewall tool with similar behavior though different syntax.

By the way, you may be wondering how Apache Web Server and some other web servers solve the Catch-22. My understanding is this: They pull a trick when they launch. During the launch process, the app temporarily acts as superuser to grab access to port 80, then immediately switches back to being a normal user to complete launching.

Use at your own risk. Don't blame me, sue me, or hate me. But you may send me Peet's Coffee gift cards.


REALbasic has a "for each" syntax

I read this tip about the REALbasic language having a convenient "for each" syntax to access the elements of an array.

Here is an example pulled from my code in a REAL Studio 2010 project. My window has an array of threads. I want to kill all of those threads when the window closes.

// Close event handler on my Window

  dim t as Thread

  for each t in self.usageReporterThreads

    if( t <> nil ) then

      t.Kill // Kill each thread in the array.

    end if


By the way, that's an example of polymorphism. The array holds instances of a subclass of Thread. But when I retrieve each instance, I can consider it as "Thread" (the parent class). The method I need ("Kill") is defined as part of "Thread" so there is no need to make this code block aware of the subclass. If I change the subclass, this code continues to work (does not break or need updating).

Here's a generic example of this "for each" syntax:

for each x in someArray

  if x <> nil then

    // Do some work.

  end if



"Shell" in REALbasic to access command-line

REAL Studio 2010 offers the "Shell" class to call command-line tools.

Any command you would call in the "Terminal" program in Mac OS X or Linux, or the "Command Prompt" program in Windows, you can programmatically call from your REALbasic code. Any text that would otherwise appear at the console in those programs is returned to your code as a String.

Here's a simple example. If you have installed a Java implementation on your computer (already built into Mac OS X), the "java" command on the console will invoke the Java Virtual Machine. Adding "-version" generates a few lines of text describing what version of the Java platform is installed. Executing "java -version" makes a fine demo of the Shell class. Put the following code on a button's Action event handler.

Dim s As New Shell
dim commandLine as String = "java -version"
s.execute (commandLine)
msgbox s.result

You should see a dialog like this:


Format command

I built a little project to let a programmer new to REALbasic experiment with the "Format" command in REAL Studio 2010 Release 3.1.



Database connection example code for REALbasic.

Here's example REALbasic code for making a connection to REAL Server 2009. The docs have similar code, but this is a bit more complete.

You can copy this code into a PushButton's Action event handler for REAL Studio 2010. Also works in a Web Application if you are using the newly announced Web Edition, in which case you put this code in the Clicked event handler of a Button.

This example assumes a TextField on your form named "dbStatus".

You can make some helper methods to assist with chores such as logging problems, but this code will at least get you started.

dim db as new REALSQLServerDatabase

db.Encryption=128 // Always encrypt your connection when on an untrusted network.
db.UserName="YourUserNameGoesHere" // "admin"
db.DatabaseName = "YourDatabaseNameGoesHere"

if( db.Connect ) then
self.dbStatus.Text = "Success connecting to db server"
else // Error connecting
self.dbStatus.Text = "Error occurred when connecting to database server. Code: " + str(db.ErrorCode) + " " + db.ErrorMessage
end if

dim sql as String = "select * from YourTableGoesHere;"
dim rs as RecordSet = db.SQLSelect( sql)
if(db.Error) then
self.dbStatus.Text = "Error when running a SELECT. Code: " + str(db.ErrorCode) + " " + db.ErrorMessage
else // Else normal.
if( rs = nil ) then
self.dbStatus.Text = "ERROR - RecordSet is nil"
self.dbStatus.Text = "Record count = " + str( rs.RecordCount ) // Replace with more useful code.
end if
end if


// ------------------------------------------
if( db <> nil) then
if( db.IsConnected ) then
end if
end if

I wish I could delete a blog entry.

'nuf said.